Using AI as a Thinking Partner for Large-Scale Engineering Systems

Most discussion of AI in software engineering assumes the bottleneck is typing. Julie Qiu's argument is that for anyone leading a large system, the bottleneck is holding context: understanding what a decade-old, multi-team, multi-language system actually does before you can responsibly redesign it. Her framing is that she did not need something to write code for her, she needed "more RAM." The talk is a field report on using AI to get that RAM, and an equally careful account of where it failed.

Qiu is a Senior Staff Engineer at Google and the Uber Tech Lead for the Google Cloud CLI and SDK. Before that she spent several years on the Go programming language team, where she led the Go security team and worked on Go's vulnerability management support and pkg.go.dev. She presented this 42-minute talk at QCon AI New York 2025; InfoQ published the recording, slides, and transcript on May 15, 2026.

These notes report what Qiu presented. Where I add background that an intermediate engineer needs but the talk did not state, the text says so explicitly.

What You Will Learn

• Why a system can be individually simple in every part and still be incomprehensible as a whole, and what that does to a technical leader.
• The five distinct roles Qiu found for AI, and the concrete workflow she used for each rather than the slogan.
• How to use a coding agent as a code archaeologist across hundreds of repositories without trusting its output blindly.
• How git worktree plus a design-only branch turns an AI agent into a low-cost architecture simulator for cross-language design questions.
• Specific quantitative results she reported, including a 3,600-line configuration deletion, and how much weight each deserves.
• The consistent failure patterns of AI-authored Go code, and the categories of work where she found it reliably excellent.
• Why two of her three redesign attempts failed, and what those failures teach about scope, abstraction, and engineer buy-in.
• Which parts of design and review work remained stubbornly human, and why that is a structural property rather than a temporary model limitation.

The System: Nine Languages, One API Surface

To understand the problem you need the shape of the system. Qiu's team builds the tools developers use to interact with Google Cloud: the gcloud CLI and client libraries currently offered in nine languages. These are what developers use to authenticate, call APIs, manage resources, and automate deployments, so they largely define the Google Cloud developer experience. The same operation, creating a storage bucket, appears as a gcloud command, as Python code, and as Go code. The Python and Go versions are client libraries: language-specific wrappers around the API that make it natural to use in the language of your choice.

On paper the production process is simple. Service teams such as Storage, Pub/Sub, and BigQuery own their services and define the API surface: the parameters it accepts, the methods it exposes, and the data it returns. They describe that API in a shared specification format. Qiu's team takes those specs as raw material for generators, runs the generators for all nine languages, layers on authentication, veneers that make the result nicer to use, cloud features, product features, and language features, and publishes the output to each ecosystem's package manager: PyPI, npm, Maven, and so on.

flowchart LR
    ST["Service teams
(Storage, Pub/Sub, BigQuery)"] --> SPEC["API specs in a
shared specification format"]
    SPEC --> GEN["Per-language generators
(nine languages)"]
    CFG["Language-specific
config and glue"] --> GEN
    GEN --> LAY["Layered on: auth, veneers,
cloud, product, language features"]
    LAY --> PUB["Publish to package managers
(PyPI, npm, Maven, and so on)"]

Underneath, Qiu says, it is genuinely complicated. A multi-language system that evolves for multiple decades across multiple teams accumulates inconsistencies organically. Her key observation is that each difference was probably small when it was made and entirely logical for the context it was made in. The result is a system that is hard to see, and her precise diagnosis is worth quoting in substance: the problem is not that any one area is hard, it is that the system is too big. Too big to fit in a single document, too big to draw on a whiteboard, too big to pack into a single human brain.

When she joined two years before the talk, her first move was to gather evidence. She started an intake process asking her team to file bugs, friction, and ideas, on the assumption that if all the problems and historical context sat in one place, a solution would emerge. The intake worked in the narrow sense: it showed how much her team dealt with daily. What it could not show was the shape of the system that all those edge cases fit into. She could sense a blind spot but could not bring the picture into focus. Her summary of the state she was stuck in is precise: she knew which pieces mattered, she just could not put them together.

The Bottleneck Is Context, Not Keystrokes

Qiu is a TL in an organization of over 70 people, spanning more than 400 repositories. Her reframing is the pivot of the whole talk. Conversations about AI in software normally center on developer productivity: moving faster, code assistants, autocomplete. None of that addresses her constraint, because her bottleneck was never typing. What she needed was a way to hold the state of everything happening across the org and those 400+ repositories, with the relevant pieces surfaced at the moment she needed them, so she did not have to hold it all in her head simultaneously.

From working that way over the past years, five distinct roles emerged for her. They are worth reading as a sequence rather than a menu, because each one depends on the output of the previous one:

The Hypothesis, and Two Failed Attempts

Before involving AI, Qiu had a hypothesis: this could all be simpler. Instead of every language team implementing its own build, test, and release pipeline, she wanted to collapse the whole thing into one production pipeline with one CLI tool, one release process, one technology stack, and one configuration system. The stated design goals were elegant enough to unify all workflows, simple enough that language teams could focus on the language-specific work that delivers product value, and flexible enough to differentiate where product and language features genuinely required it.

The first attempt was what some people call a steel thread: design the entire system end-to-end for a single language, go deep, and make it work. Within a few months two problems appeared. The team was taking on too much, and the system started shaping itself around the one language they had picked. The core idea still looked promising, but migration would take years.

The failure she says she did not fully appreciate at the time was not technical. Technical design in this domain is tightly tied to the philosophies of the engineers behind it. An engineer working in a specific language ecosystem cares deeply about that ecosystem's details, and sometimes those details do not actually matter at the architectural level. Giving engineering teams flexibility, buy-in, and the feeling of being heard was, in her account, a critical component of making the project work at all. Her reasoning is empathetic rather than political: as an engineer, she might do something because someone told her to, but she is much more likely to commit if she believes the person understood the challenges she was dealing with.

The second attempt narrowed scope: a thin slice of the system across two languages simultaneously. The details became overwhelming again. Generalizing too early produced abstractions so generic that state became unmanageable outside the language-specific areas. A second, more mundane constraint hit as well. They had chosen Go as the implementation language for the pipeline, and the team assembled for the project came from different language ecosystems, so teaching them Go became its own bottleneck. They restarted again.

Supplementary note, not from the talk: the "generalize too early" failure is the standard tension behind the rule of three, the heuristic that you should see a pattern at least three times before abstracting it. Qiu's version is stronger, because with nine divergent implementations the third example may still not be representative.

After hundreds of design docs, a lot of code, and many PRs, she reset to first principles. She traced how they had arrived at the current state, then opened a terminal, created an empty experimental directory, and wrote a README.md containing the simplest possible version of the system in her head. At its core the CLI had three responsibilities: read and manage state and write it out when necessary; take the specs and configs and produce a client library; and release and publish that library to a package manager.

She pointed Gemini CLI at that file and asked it to build what she had described. With one prompt it did: CLI framework, flag parsing, command structure. Her assessment is the important part. That was the boring part, and without a migration strategy the tool is just a toy. She could not apply a new tool to the ecosystem she wanted to change without first understanding that ecosystem.

AI as an Archaeologist

In a system built over many years, documentation is often stale or incomplete, and the only thing you can really trust is the code that is actually running. Reading 400+ repositories by hand is slow, manual, and mentally exhausting.

She started small, with a single repository. She knew the Python repository held the configs for the majority of their setup in one place, so she cloned it locally, opened Gemini CLI, and asked how the Python generator actually works: what it takes as input and what it produces as output. In a single prompt the model distilled thousands of lines of code down to essential behavior: inputs, process, outputs. It surfaced files she did not know existed and mapped the generation work end-to-end. She estimates this would normally have taken her weeks.

She pushed further and asked what it would look like to consolidate all the files it had found into a single YAML file. It produced one, probably over 10,000 lines. She is clear that this was not a pretty answer, but it was a starting point, and its value was the questions it provoked: what are all these regexes doing, can some of this metadata be derived from other fields, do we really need to hardcode all these strings, which parts of this API configuration are genuinely Python-specific, and how much of it is duplicated across nine languages and therefore belongs in a language-neutral layer.

Where the Archaeologist Lied

Qiu is direct about the failure modes, and this section is the most useful calibration in the talk. The answers were not perfect. Sometimes she had to stop and ask whether a folder it described actually existed. Her recurring joke is the model's response pattern: "of course, I was absolutely right," followed by an admission that the folder does not exist and a correction. Sometimes she had to supply more context, pointing it at a GitHub issue or a specific file and asking whether that changed its analysis.

The concrete error taxonomy she reports is: omitting files, omitting parameters, inventing files, and inventing parameters, with high confidence every time. Her most important observation is about detection latency. She did not notice some of these errors until she started prototyping and things simply did not work. This is the practical argument for treating archaeology output as a hypothesis that must be executed against reality rather than as documentation.

Supplementary note, not from the talk: the sycophancy Qiu keeps mocking is a known and measured behavior in instruction-tuned models, and it is dangerous precisely in this workflow, because the tool agrees with your framing at the exact moment you are trying to discover that your framing is wrong. Asking a model to argue against a proposal, as Qiu later does in the critic role, is a partial mitigation.

From Per-Language Notes to a Map

After hours of back and forth she had learned a great deal but was not sure what to do with it, so she asked the model to write the conversation down into a python.md file. Then she repeated the exercise for Rust: how the generator worked, what configuration it used, its inputs and outputs.

Rust turned out to follow a completely different philosophy, and the contrasts she lists are concrete:

Repeating this for every language made the pattern clear: everyone was solving the same high-level problem, but the implementations had diverged dramatically over time. AI spared her weeks of repetitive labor installing every generator, navigating each codebase, and reading old docs, and let her reconstruct much of the logic instead.

The output was a folder of Markdown files, one per language, each summarizing how that pipeline actually worked. Reading across them gave her something she had not had before: a map showing what was the same and what was different in the shape of each system. That map resolved the system into three concrete pieces.

flowchart TD
    A["Service-team component
Product-specific API surface"] --> P
    P["Platform component
Language-agnostic infrastructure"] --> L
    L["Language component
Idiomatic developer experience"]
    X["Actual org structure:
language teams owned all three"] -.-> A
    X -.-> P
    X -.-> L

The service-team component was specific to the products those teams were creating. The platform component was infrastructure that was language agnostic, where the implementation language did not matter. The language component was where language-specific knowledge genuinely mattered to produce an idiomatic developer experience, and was the only part that required language-team expertise.

What they had actually done was let the language teams own everything. Her conclusion is organizational: the org structure was what caused the differences. Comparing patterns across languages let her finally see what was genuinely different versus what only looked different because of history.

Supplementary note, not from the talk: Qiu does not name it, but this is Conway's law, the observation that system structure mirrors the communication structure of the organization that builds it. My own framing of what follows is the "inverse Conway maneuver," changing ownership boundaries so the desired architecture becomes the natural one; Qiu describes the three-component map and the ownership diagnosis, not that remedy by name.

AI as an Experimenter

With the map in hand, the unified pipeline felt far more feasible than in the earlier attempts. Qiu draws a sharp distinction here: until this point AI had been digging up the past and helping her understand what exists, but design does not happen in the past. Design happens in what you imagine, and in making hard trade-offs about what you want the system to be.

The economics of testing a design idea were the obstacle. If she had a question about Python and wanted a prototype, she would pull someone off the Python team for a week. Seeing the same idea in JavaScript meant asking another manager for the same staffing. Doing that across nine teams, and tracking what all those engineers were doing, is a large coordination cost to answer one question: is this idea even viable? Her framing of the asymmetry is the crux: you cannot ask a team of engineers to read hundreds of thousands of lines of code just to check whether a pattern exists, but you can ask AI to do exactly that. AI became a low-cost simulation engine for architecture.

The Working Setup

Her workflow is specific enough to copy:

• A design-only main branch. On main she talked to Gemini purely about design, with an explicit instruction not to write any code and to just talk with her.
• Parallel language branches via git worktree. She used git worktree to set up three directories and multiple branches on the repository, one each for Python, Go, and Rust, chosen because she knew their real constraints differed substantially. Separate worktrees let her experiment freely in every language without the branches interfering with each other.
• A reusable prompt.md role definition. She wrote a prompt assigning the model the role of design partner, recording where all her artifacts lived, which alternatives she had already considered, and the to-do list she was working through. Every new session or tab could read that file to recover context. On a language branch she would tell it to read prompt.md, absorb what she was trying to do, then implement it in that language.

Supplementary note, not from the talk: git worktree attaches multiple working directories to one repository, each on its own branch, sharing a single object store. That matters here because each worktree can hold its own agent session and its own build artifacts without the constant branch switching that would corrupt an agent's assumptions mid-task.

The point she stresses is that the generated code was not good, and it did not need to be. What mattered was that attempting an implementation exposed exactly where the design was underspecified, where it was awkward, and where the model had hallucinated ideas for her.

Four Experiments and What They Actually Proved

Redundant version fields. She noticed the global config file recorded all the versions, and so did the language-specific files: version.py for Python, version.go for Go, Cargo.toml for Rust. She asked Gemini whether the version field should be removed from the YAML config given it already existed in those files. Of course she was absolutely right: single source of truth is better, remove the duplication. Implementing it revealed that some of those files were themselves generated from the global config, and not uniformly, which is why she had not spotted it by reading. The experiment made the detail impossible to miss and raised a better question: why are some version files generated and others constructed manually?

Inferring structure from names. She spotted an apparent relationship between a library's name, its output folder, and its API path, and asked how many cases could be inferred from the name alone. The answer was that the name, API path, output directory, and the YAML configuration file followed a common pattern across all languages. For a library named grafeas, the API path was grafeas/v1, it lived in packages/grafeas, and the config file was grafeas_v1.yaml. Because they already had a repository containing all the API specs the service teams wrote, she could write a script to traverse that directory and infer these values instead of maintaining a list by hand, eliminating roughly 100 entries from every language's config file.

One big file or many small ones. She held a strong prior that one 4,000-line config file could not scale as Google Cloud added more APIs, and that Rust's one-file-per-crate design was cleaner. The experiments reversed it. Because so many fields were redundant, once the redundancy was removed a single file was better: patterns became obvious, duplication became visible, and search and replace became trivial. She flags this explicitly as counterintuitive and as an important assumption the experiments changed.

Positional arguments versus flags. generate originally took a name flag, but since generation was required across all libraries the flag was unnecessary. Her process for a change like this is the reusable part: update README.md, then tell Gemini on each language branch to rebase, update everything, and rerun all the experiments. That kept the core design in sync while she worked through per-language details, and more importantly flushed out subtle inconsistencies, separating the places that merely sounded good because the model kept agreeing with her from the places she was actually right.

She also used pattern matching to confirm that the legacy configs could be migrated to the file format she was proposing at all, which is a viability check rather than a design question.

What the Experiments Surfaced by Accident

Two structural discoveries came out of the process rather than being targeted. Every language generator produced client libraries but also produced other files: some emitted READMEs, some changelogs, and their contents differed slightly because they lived in language-specific generators. Standardizing those would improve the user experience and reduce maintenance. The same divergence existed on the input side: some language teams had service teams maintain a YAML file of language-specific hints, some used Bazel targets, and some had bespoke glue. Her recurring judgment applies again, that every individual choice made sense in context, but viewing the whole picture makes it clear the files can be eliminated or condensed into a single source of truth.

The benefit she calls unexpected is social. The experiments let her have far more interesting conversations with her team, far earlier than she otherwise could, because AI got the ideas to a state worth sharing. That human feedback showed where her mental model matched reality and where it did not, and it exposed places where AI had hallucinated and convinced her she was absolutely right, at which point her team was understandably confused by what she was proposing. AI sketched enough to answer simple questions such as whether the workflow works and whether she was missing something obvious; involving the team earlier is what moved the design forward.

AI as a Critic

At this stage she had a user guide, a design doc, and a set of principles. People could react to the big ideas, but nobody was going to sift through every individual design decision Gemini produced at two in the morning. That required a different kind of feedback, so she inverted the prompt: instead of asking what this could look like, she asked where she was wrong and where this could go wrong. The formulation she calls most insightful is asking which parts of the design look overengineered and which parts seem unnecessary.

The win. The Go config carried a remove_regex field on almost every line. She asked whether it could be removed and whether it was always the same. The model examined 185 libraries and found that 179 followed a completely predictable formula, with only six requiring genuine Go-specific config. That let her delete 3,600 lines of configuration.

The failure. She asked it to help simplify the release workflow, proposing a single release command that commits, tags, pushes, and publishes, with defaults and flags to skip steps. Of course she was absolutely right, and the model pointed out this is exactly what cargo release does, a tried-and-true path. Her engineers immediately produced the questions that mattered: does it publish by default, or does the execute flag also create a tag? If the publish flag fails, how do you retry without retagging? Why is publishing off by default when publishing is the entire point of the command?

Her analysis generalizes well beyond release tooling. AI offered a textbook solution that sounded simple on paper, but in the context of a team sharing a release workflow the simple interface made things more complicated, because now everyone must understand how the defaults interact instead of running separate commands one by one.

What the critic role delivered was not correct answers. It consistently forced her to question decisions and justify what she was doing, and she says that scrutiny was the real value. By the end of the phase she had a design that had been interrogated from multiple angles, simplified through critique, and refined to a point she could stand behind, all before involving dozens of teams and months of engineering.

AI as an Author

Her first rule is that you cannot say "write the whole thing," at least not if you want something you can show to someone and maintain long-term. Like delegating to any person, you must be specific about the ask, break the task down, and expect to review the result.

Consistent Failure Patterns in AI-Written Go

• Narration disguised as documentation. Writing code to read and write a YAML package, the model told her it was defining the input and output file names, reading the YAML file, and unmarshaling the data. None of those comments said anything the single line below did not. Her point is subtler than "noise": because the comments restate syntax, they make the code harder to read, since she has to filter them out to see the structure.
• Whitespace that destroys grouping. In a switch statement whose cases each simply ran the generator for a language, it inserted blank lines between every case and function. That broke the visual grouping Go developers rely on and made conceptually identical cases feel unrelated for no reason.
• Defensive nil checks in the wrong places. A createLibrary function returned nil when its input was nil, then called a helper that performed the same check again. Her critique is about contracts, not redundancy: the duplicated check blurs the API contract, because it is no longer clear whether the helper is supposed to accept nil at all. Her diagnosis is that the model does not understand where safety belongs, so instead of establishing a clear invariant once, it checks for nil everywhere.
• Forgetting standard tooling. It consistently failed to run gofmt or goimports, producing ungrouped imports and wrong spacing, and she had to remind it. She emphasizes that these are not personal style preferences: the specs are well defined, the tooling is public, and the format is standardized. Her conclusion is the sharpest line in this section, that the output looked like Go and kind of was Go, but did not reflect the structure Go developers use to communicate with each other.

Where It Was Genuinely Excellent

Alphabetizing struct fields, which she happily admits she finds tedious. Refactoring imports when moving a package, renaming every reference across the repository without her having to remember the right command. Fixing compile and linter errors: when adding a parameter to a function she would normally run go test, collect the compile errors, and hand-edit every file, but instead handed the errors to Gemini, which updated the call sites, propagated the parameter, and got everything compiling again.

The pattern she extracts is the transferable rule: give it something concrete, repetitive, and with explicit correctness criteria, and it is an amazing accelerator.

She also found that supplying a style guide dramatically improved output. She passed in documentation from the Go website, including the Effective Go manual, the Go commenting and formatting guides, and the structured logging blog post, and instructed it to follow those conventions when writing code. The quality improved substantially.

Her overall characterization is that this was the work she would previously have done herself while watching television or listening to a podcast. AI does it in seconds and she reviews the result. She is explicit that she still wrote the important logic herself; AI let her reach the same quality on the small repetitive tasks in a fraction of the time.

AI as a Reviewer

A common complaint about AI-assisted development is AI slop: generating code and dumping the review burden on a human. Qiu wanted to be deliberate about not doing that, so she ran the gemini-code-assist plugin as part of her workflow to check her PRs before humans saw them.

The catches she reports span several categories. A missing error check where a file might be present but not readable, which would have caused a test to pass silently when it should have failed. A structural suggestion to use a table-driven test. A catch that the other AI had not run the Go formatting tool she wanted. A suggestion to use a constant instead of hardcoding the number three in several places, keeping them in sync. And, in code she wrote while half paying attention to a meeting, code that compiled and looked plausible, two flagged bugs she fixed before a teammate had to be confused by them.

Where AI Stopped Working: What Humans Caught

The limits section is where the talk earns its thesis, because every example is a review comment AI could not have produced.

A PR of hers compiled and passed tests, and a teammate immediately asked why an apparently unrelated change was bundled into it. Scope and intent are not visible from correctness.

On a PR flattening the YAML structure, AI gave useful mechanical feedback, and a teammate said that having a duplicate function made him "really sad." Qiu's read on that word is the sharpest observation in the talk: sadness is intuition talking. Both versions worked perfectly fine, and the feeling came from knowing what good looks like.

The same reviewer asked where the project was going, given that she had a field called language but had also said they might use this for gcloud, so is gcloud a language? Her generalization is that when engineers review your PR they are not only thinking about the PR in front of them, they are thinking about the roadmap and where things are heading. AI could not answer questions about things outside the context it had been given.

A third example is a pure trade-off discussion. One teammate proposed consolidating several sources in a struct into a map to reduce typing. Another argued for keeping the explicit fields for now, because the project was still in the design phase and explicit fields would make restructuring easier later. That decision requires knowing the project's context, what they were still trying to do, and which kinds of mistakes they wanted to keep easy to catch.

Architecture And Data Flow

The complete loop she describes, from a system nobody could see to a design she could defend, looks like this:

flowchart TD
    A["400+ repos, stale docs,
a decade of divergence"] --> B["Archaeologist:
ask the agent how each
generator actually works"]
    B --> C["One Markdown summary
per language"]
    C --> D["Map: service / platform /
language components"]
    D --> E["Design-only main branch
plus prompt.md role file"]
    E --> F["Experimenter:
git worktree branches for
Python, Go, and Rust"]
    F --> G["Implementations expose
underspecification and
hallucination"]
    G --> E
    F --> H["Critic:
where am I wrong?
what is overengineered?"]
    H --> E
    E --> I["Share early with the team"]
    I --> J["Humans catch: hallucinations,
roadmap fit, taste, trade-offs"]
    J --> E
    E --> K["Author + Reviewer:
production code with
gemini-code-assist on PRs"]
    K --> L["Human review
for scope and intent"]

The important property is that every AI-driven arrow feeds back into a human decision point. Nothing in her workflow ends at a model's output.

Trade-offs And Limitations

• Archaeology output must be executed, not merely read. Qiu did not detect several of the model's omissions and inventions until prototyping failed.
• Sycophancy is the dominant risk in a design conversation. See the note in Where the Archaeologist Lied; the practical consequence is that she only caught the worst cases by showing the design to her team.
• AI-proposed simplifications ignore team dynamics. The single release command was defensible in isolation and a poor fit for a team that has to reason about defaults, retries, and partial failures.
• The quantitative results are one team's measurements. The 3,600 lines of deleted configuration, the 179-of-185 pattern match, the ~100 entries per language config, and the 10,000-line consolidated YAML are specific to Google Cloud's SDK repositories and should not be read as general expectations.
• Generated code violated ecosystem conventions consistently. Over-commenting, whitespace that broke grouping, scattered nil checks that blur invariants, and skipped gofmt/goimports runs. Qiu's countermeasure was to feed in authoritative style guides, which helped substantially but did not remove the need for review.
• The project is still ongoing. Qiu presents this as work in progress, not as a completed migration with measured outcomes, so the design's success is not yet demonstrated.
• Organizational change was the harder half. Both attempts failed partly on scope and abstraction; for the first attempt in particular she attributes a critical share to engineer buy-in and to language teams' legitimate investment in their ecosystems' details. The second attempt failed on premature abstraction and on the cost of teaching Go to a team drawn from other ecosystems.

Supplementary note, not from the talk: Qiu does not discuss the confidentiality implications of feeding proprietary source and configuration into an AI tool. Any team copying this workflow on non-public code needs to confirm the data handling and retention terms of the specific tool and tier they are using before pointing an agent at internal repositories.

Practical Takeaways

• Diagnose your actual bottleneck before reaching for a code assistant. If the constraint is holding context rather than producing keystrokes, optimize for retrieval and synthesis, not autocomplete.
• Use an agent to summarize each subsystem into its own Markdown file, then read the summaries side by side. The comparison across files, not any single summary, is what produced Qiu's map.
• Prototype the agent's claims immediately. Hallucinated files and parameters surfaced when she tried to build against them, not when she read the analysis.
• Separate design conversation from code generation. Keep a branch where the agent is explicitly told not to write code, and do implementation on other branches.
• Use git worktree to run parallel per-language or per-approach experiments so agent sessions do not interfere with each other.
• Write a prompt.md role file that points to your artifacts, records alternatives already rejected, and holds your to-do list, so every new session starts with context instead of your retyping it.
• Ask the inverse question deliberately. "Where am I wrong," "where could this go wrong," and "what looks overengineered" produced better results than "what could this look like."
• Delegate to AI only work that is concrete, repetitive, and has explicit correctness criteria: alphabetizing, import refactors, propagating a new parameter through compile errors, mechanical renames.
• Feed the ecosystem's authoritative style guides into the tool rather than correcting output repeatedly.
• Run an AI review on your own PR before a human sees it, specifically so that you are not offloading mechanical cleanup onto teammates.
• Look for redundancy that can be inferred rather than maintained. Naming conventions, derived paths, and predictable regexes are prime candidates, and an agent is well suited to checking a hypothesis across hundreds of repositories.
• Treat the component ownership map as the real deliverable. Qiu's structural insight was that language teams owned platform and service concerns they should never have owned; the tooling redesign follows from fixing that.
• Share half-finished designs with the team earlier than feels comfortable. AI lowers the cost of reaching a shareable state, and human review is the only thing that catches hallucination, roadmap misfit, and taste.

Key Terms

• Client library — A language-specific wrapper around an API that makes it natural to use from that language, generated in Qiu's system from a shared API specification.
• Steel thread — A thin end-to-end implementation through an entire system, used to prove the whole path works; Qiu's first attempt used one language as the thread and the design started bending toward that language.
• Veneer — A hand-written layer added on top of generated API code to make it more ergonomic or idiomatic for the target language.
• git worktree — A Git feature that attaches multiple working directories to one repository, each checked out to a different branch, sharing a single object store.
• AI slop — Low-quality machine-generated output pushed onto human reviewers; the outcome Qiu deliberately designed her review step to avoid.
• Sycophancy — A model's tendency to agree with the user's framing; represented in the talk by the recurring "of course, I was absolutely right."
• Table-driven test — A Go testing convention where cases are defined as a slice of structs and executed in a loop, suggested by the AI reviewer on one of her PRs.
• gofmt / goimports — Standard Go tools that enforce canonical formatting and import grouping; the model consistently forgot to run them.
• Single source of truth — Storing a fact in exactly one place so it cannot drift; the principle behind removing duplicated version fields and consolidating per-language configs.

Closing Assessment

Qiu's own summary is that she started this work assuming the hardest part would be figuring out how the system worked across all languages, components, workflows, products, and a decade of drift. That was genuinely hard, and AI made it substantially clearer. The hard part turned out to come after: deciding what to keep, what to change, and what to throw away. AI helped her shine a light into the corners and map what was wrong, what was redundant, and what existed only because it had always been that way, but it could not replace the messy and hard-to-articulate work of understanding, synthesizing, and deciding.

Her role-by-role verdict is consistent. The archaeologist reconstructs but cannot tell you why a decision was made. The experimenter bootstraps ideas but cannot validate them without a real team. The critic finds differences and inconsistencies but does not replace teammates' judgment. The author writes code but relies on your experience to judge whether that code is good. The reviewer catches many mechanical bugs, but humans caught the things outside the code: the context, the roadmap, and the direction the project was heading.

The framing she closes on is the one worth carrying. She initially thought the problem was that she needed more of herself, more hours and more hands. Cloning herself would not have solved it, because the bottleneck was never her hands or typing speed; it was judgment, understanding, and the messy synthesis required to decide what matters. AI cannot give you that. In her words it cannot replace your judgment or taste, and it cannot replace "the intuition that you've earned from spending years in a domain." Her stated reason is that models are trained on what already exists, on the average of the past, and in her view the most interesting engineering problems do not live there.

What AI can do is help you think, challenge your decisions, ask sharp questions, and push back like a curious teammate, and it can only do that because you have the expertise to know what to do and where you are trying to go. What it gave her was the thing she had always wanted: "those extra set of hands to do the work that was too rote to be interesting but too context heavy to delegate." That was work she knew how to do; it was simply not what she wanted to spend her limited time and energy on. Her closing metric is deliberately not a speedup: AI did not make her ten times faster, it made her ten times more present in the work, giving her leverage over the parts that became mechanical so she could reinvest that attention in the parts that were still growing. The real value she names is "freeing yourself to spend your time on the things that truly excite you and only you can do."

Reference: Julie Qiu, Using AI as a Thinking Partner for Large-Scale Engineering Systems, QCon AI New York 2025, published by InfoQ on May 15, 2026.