Rendered at 23:51:49 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
wesselbindt 5 hours ago [-]
So, I get the idea of having a layer of abstraction between your tests and the system under test. You decouple _what_ is being done from _how_ it is done, which leads to robust, easily maintained, and even more re-usable test suites. I've seen it in action, it's a great idea, I've reaped the benefits myself. Great stuff.
What I've never really understood is why some people choose natural language for the abstraction. It just seems like such an odd and expensive choice. Most of the layers of abstraction we add in our codebase are written in the programming language of choice itself, they're functions, classes, what have you. But for this one specific flavor of abstraction, folks reach for natural language.
The one potential benefit I could think of for this over a more standard DSL is that non-technical colleagues would be able to read or even write these specs. But in practice I've only seen this pan out once in my life, and that was with a more traditional DSL rather than natural language.
Is there a hidden benefit I'm overlooking?
MoreQARespect 1 hours ago [-]
I don't think so. Natural language is vague and repetitive.
I used hitchstory (YAML based stories) for doing this coz it's terse and typed.
I've tried using cucumber before and it reminds me of COBOL - which also had the same idea of making code more accessible via a "natural language-ish" interface.
righthand 4 hours ago [-]
Because the intent has always been that Product delivers the BDD statements not Engineering, but Product people have (for the most part in my experience) refused to deliver what BDD flows need to be tested. That’s why most of the time the test suites aren’t as resilient because the engineer is guessing at which flows to test.
In that context it makes more sense that Llms are a push to replace Product people not Engineering people.
scresswell 3 hours ago [-]
[dead]
jesol 7 hours ago [-]
I also think BDD is a really powerful tool for UI acceptance tests! I've now worked on adding it to two AI codebases (one engineered, another vibecoded), and so far it has been mostly positive. Having non-engineers be able to see what effects their code changes have in the UI is nice; and I find having high level API and UI tests is forcing the first layer of cleanup (standardize access to DB through models/services, and a single API SDK).
I am also really interested in standardizing the REST interface with typespec, and the database schema with DBML. I think creating these closed-loops with high-level specs is one of the best ways to make an AI project ownable by a team.
stevefan1999 6 hours ago [-]
Yep, I've been experimenting Playwright with CucumberJS and Deno for the last 3 months and it's super effective
skybrian 5 hours ago [-]
How are you doing that? Is the Playwright usage a hidden detail or do images appear in the spec?
sroerick 2 hours ago [-]
I hadn't heard of BDD before, but the idea of "executable spec" is certainly very interesting
lifeisstillgood 8 hours ago [-]
The example of “transcibe the initial project discussion, turn that into BDD and that makes the rails to guide the AI is … either brilliant and insightful or a snake eating it’s own tail.
I think it has to be worth a try though…
jaggederest 6 hours ago [-]
I've been doing that for almost a year and I feel like I'm taking crazy pills when I talk about it, because it works so well.
I transcribe most product meetings, turn them into a transcript, run that through an accuracy pass with a bunch of context (mostly vocabulary definitions etc), and use that as a starting doc for generating issues and tests. There are apps that do it fully end to end now I think.
lifeisstillgood 5 hours ago [-]
Any hints for running pilot projects (at large, bureaucratic org)
jaggederest 5 hours ago [-]
Yeah, basically I would just start transcribing everything, if you aren't already, and get a bunch of context. It's best to start from the very beginning but the reality of enterprise is that nothing is truly greenfield.
Maybe start with a project kickoff or brainstorm, some kind of planning meeting, and select one task out of it to start. I always start with just the meeting transcript as a context, and expand context from there as needed with things like internal standards docs, libraries, references from other projects, etc, and only as little as is necessary. That generates the project plan, and then I feed that into claude code or whatever.
I think the biggest thing I've done recently is start to be extremely critical about the output, basically using promptfoo to try to figure out what the right level of context and instructions are, not just for external facing e.g. chatbots or whatever, but for my own internal development process, planning, everything. The more you can pin down and calibrate the better - it's easy to run a given prompt for a month, so spending a day (or even a week!) out of the month to refine your prompts beats just grinding ad hoc.
Edit: another thing I do is generate tests bdd-style separately from the code that they're testing (for e2e and high level at least), that derives out of the project plan too.
krautsauer 9 hours ago [-]
"Behaviour Driven Development"
I was expecting something on Binary Decision Diagrams. SAOD is worst with TLAs.
WalterGR 9 hours ago [-]
What is SAOD? Do you mean SOAD?
contradictioned 9 hours ago [-]
Severe Acronym Overload Disorder.
Funny enough, Google's overview box also suggests soad: "SAOD (or SOAD) most commonly refers to the rock band System of a Down"
WalterGR 8 hours ago [-]
Thanks. (That’s what it suggested for me as well. Clarifying that I meant in the context of software, it suggested a misspelling of SOAD - Service-Oriented Analysis and Design.)
8 hours ago [-]
bossyTeacher 8 hours ago [-]
This is probably HN's tech background speaking. In the enterprise dev world, everyone knows what BDD means just like every dev out there knows what OOP means.
scresswell 10 hours ago [-]
Yadda 3.0.0 is out. The release modernises the JavaScript BDD library, but more interestingly, it was largely built by Claude Code and points to why executable specifications may become even more valuable in an agentic development world.
ur-whale 9 hours ago [-]
One suggestion: my bet is that close to no one knows what the acronym BDD stands for. The first comment on this post is in fact exactly about that.
I'd consider explaining what BBD is at the start of the front page of your project.
davepeck 7 hours ago [-]
It may be an age thing. 15-ish years ago, TDD, BDD, Red/Green Testing, and a whole bunch of others were hot topics. They’re still useful approaches to know about, even if some (like, IMO, BDD) didn’t really stand the test of time.
cbsmith 8 hours ago [-]
"my bet is that close to no one knows what the acronym BDD stands for".
That's disconcerting. Software engineers ought to know.
ashleyn 7 hours ago [-]
As a woman my first thought was Body Dysmorphic Disorder.
citizenkeen 8 hours ago [-]
I’ll take that bet, since we ask about it in our interview questions.
jaggederest 6 hours ago [-]
What's the hit rate? Give us some data, if you can :)
ur-whale 1 hours ago [-]
> I’ll take that bet, since we ask about it in our interview questions.
You evaluate candidates based on their intimate knowledge of acronyms?
LOL.
Pray tell where you work so we all collectively never apply for a job there.
Arubis 5 hours ago [-]
My go-to for forcing AI to at least attempt to follow BDD has been to demand that it use Phoenix Contexts.
blfr 7 hours ago [-]
Very impressive that it was done with Opus 4.8.
My experiences with Opus have been such that I always max out my Fable allotment but rarely exceed 40% of the remaining limit on Claude.
Amekedl 8 hours ago [-]
Being hard here, though:
Purely dismissing it as not being worth of my time due to the example alone.
What I've never really understood is why some people choose natural language for the abstraction. It just seems like such an odd and expensive choice. Most of the layers of abstraction we add in our codebase are written in the programming language of choice itself, they're functions, classes, what have you. But for this one specific flavor of abstraction, folks reach for natural language.
The one potential benefit I could think of for this over a more standard DSL is that non-technical colleagues would be able to read or even write these specs. But in practice I've only seen this pan out once in my life, and that was with a more traditional DSL rather than natural language.
Is there a hidden benefit I'm overlooking?
I used hitchstory (YAML based stories) for doing this coz it's terse and typed.
I've tried using cucumber before and it reminds me of COBOL - which also had the same idea of making code more accessible via a "natural language-ish" interface.
In that context it makes more sense that Llms are a push to replace Product people not Engineering people.
I am also really interested in standardizing the REST interface with typespec, and the database schema with DBML. I think creating these closed-loops with high-level specs is one of the best ways to make an AI project ownable by a team.
I think it has to be worth a try though…
I transcribe most product meetings, turn them into a transcript, run that through an accuracy pass with a bunch of context (mostly vocabulary definitions etc), and use that as a starting doc for generating issues and tests. There are apps that do it fully end to end now I think.
Maybe start with a project kickoff or brainstorm, some kind of planning meeting, and select one task out of it to start. I always start with just the meeting transcript as a context, and expand context from there as needed with things like internal standards docs, libraries, references from other projects, etc, and only as little as is necessary. That generates the project plan, and then I feed that into claude code or whatever.
I think the biggest thing I've done recently is start to be extremely critical about the output, basically using promptfoo to try to figure out what the right level of context and instructions are, not just for external facing e.g. chatbots or whatever, but for my own internal development process, planning, everything. The more you can pin down and calibrate the better - it's easy to run a given prompt for a month, so spending a day (or even a week!) out of the month to refine your prompts beats just grinding ad hoc.
Edit: another thing I do is generate tests bdd-style separately from the code that they're testing (for e2e and high level at least), that derives out of the project plan too.
I was expecting something on Binary Decision Diagrams. SAOD is worst with TLAs.
Funny enough, Google's overview box also suggests soad: "SAOD (or SOAD) most commonly refers to the rock band System of a Down"
I'd consider explaining what BBD is at the start of the front page of your project.
That's disconcerting. Software engineers ought to know.
You evaluate candidates based on their intimate knowledge of acronyms?
LOL.
Pray tell where you work so we all collectively never apply for a job there.
My experiences with Opus have been such that I always max out my Fable allotment but rarely exceed 40% of the remaining limit on Claude.