Caio Saldanha
AI Architect & Executive AI Advisor
Caio Saldanha is an AI Architect and Executive AI Advisor working at the intersection of autonomous systems and corporate governance. He architects AI solutions and behaviors within heavily regulated enterprises, and guides boards and executives through AI adoption with purpose, methodology and operational control grounded in contemporary cybersecurity principles.
Essays
42Technical debt works better as a loan book, and the missing column is the interest rate
21 min readCunningham's debt was a loan taken to learn and repaid once you know more. Taken literally, the metaphor asks for a register with principal, interest and expected life for each item, plus a payback rule for repayment.
Data leakage is how a weak model earns a strong score
12 min readA model trained on pure noise can score 86% in cross-validation if one preprocessing step is in the wrong place. Leakage explains many results that vanish on contact with production.
A CTO's no to hype needs a test, a tripwire and a log, or it is only a mood
18 min readVenture firm Bessemer publishes a page of the companies it passed on, including Google. A technology leader's refusals leave no such record, and without one nobody can tell a good no from a lucky one.
Build, buy or fine-tune is three decisions, and most debates merge them
12 min readWhere the model comes from, how it is adapted to your task, and who owns the system around it each have a different answer. Treating them as one question produces expensive mistakes in both directions.
Scaling laws forecast loss, and the forecast changed twice
12 min readChinchilla showed that the best-known training recipe spent its budget in the wrong place. The next correction is arriving from the supply of human text, and it matters for anyone planning around model progress.
Token prices keep falling, and AI bills keep rising: elasticity decides which wins
13 min readWhether a price cut shrinks or grows total spend depends on how demand responds. Budgeting AI from a price list instead of a volume forecast is the most common planning error in the category.
Inference economics shapes the product you can afford to build
12 min readA single 128,000-token conversation with a 70B model holds about 40 GiB of cache. That one number explains the price sheet, and the price sheet explains which designs are cheap.
Reinforcement learning works where an answer can be checked
12 min readReasoning models improved fastest on problems with a program that can grade the answer. That rule also tells a company which of its own tasks are ready for reinforcement fine-tuning, and which are not.
AI sovereignty for a Latin American company is a list of dependencies you can price
13 min readPortuguese costs more tokens than English, invoices arrive in dollars, and data law reaches across borders. Each dependency can be measured and hedged without waiting for a national model.
The eval set is the specification
12 min readPublic benchmarks measure the benchmark. For an LLM application, the set of cases you wrote, labeled and defended is the nearest thing to a specification you will have.
When a model solves your take-home, redesign the work sample instead of policing it
21 min readAnthropic rebuilt its engineering take-home as Claude models beat it. The selection literature treats a take-home as a work sample, so the repair is a better sample with structured scoring. I found no study that validates the new formats.
Routing between models saves money, and the exit plan is what you are really buying
12 min readRouter papers promise large cost cuts, and some deliver. The lasting value of a multi-model design is the ability to leave a provider on a Tuesday afternoon without a rewrite.
Decision rights belong where the knowledge is, and AI products need them written down
21 min readThe Challenger launch shows what happens when the right to stop, waive and accept risk is spread across people who each hold a piece of it. An AI product team can write the equivalent down in eight lines.
Hidden technical debt in machine learning, ten years later
12 min readThe 2015 Google paper warned that quick wins in machine learning are never free. Most of its categories survive in LLM systems, with a dependency the authors could not have pictured: a model someone else can change.
When prose is free, the memo worth approving names what would change the author's mind
19 min readFluent text used to cost effort, which made it weak evidence of thought. Now that models write it for nothing, reviewers of memos and design docs should ask for the decision, the rejected options, the evidence and the exit condition.
What the Transformer changed was the order of computation
12 min readAttention existed three years before the Transformer. The 2017 paper removed the sequential dependency that kept recurrent networks from using the hardware, and the rest of the decade followed from that.
Brooks's law prices coordination, and AI code generation moved the bill to review
20 min readBrooks introduced his law as an outrageous oversimplification of two costs, ramp-up and communication. AI code generation made writing cheaper and left review and integration as the place where teams pay.
Blameless postmortems are a method for getting accurate facts, and the test is what changes
21 min readBlamelessness gets accurate facts from people who have reason to hold them back. Applied to a chatbot's wrong promise or a silent model regression, it points at the gate that failed, and its test is what changes afterward.
When your supplier is also your rival: the five forces for AI applications
12 min readModel providers can cut off access, sell directly to your customers and cut the price of your input for everyone at once. Porter's framework still explains where the profit goes.
Disruption theory is narrow on purpose, and most AI products fall outside it
18 min readChristensen's theory predicts one specific thing: incumbents rationally ignore entrants who start with overlooked customers. Suite copilots fail that test, and a five-question diagnostic shows where the theory bites in AI and where it does not.
Neither tool nor junior hire: manage AI output as a production lot with a defect rate
21 min readTreating a model as a deterministic tool or as a junior employee both fail. A quality-control view works better: estimate the defect rate by sampling, set review depth by the cost of an error, and measure the reviewers too.
AI governance that ships is a set of gates in the pipeline, not a committee at the end
12 min readThe EU moved its high-risk deadline by sixteen months and Brazil's bill is still pending. Both facts argue for building governance as risk-tiered controls that produce evidence as a side effect of engineering.
When a workflow beats an agent
12 min readAgents spend reliability, latency and money to buy flexibility. Most business processes should hand the control flow to code and give the model only the judgments.
The architect who still writes code should mostly be reading it
21 min readArchitects who stop touching code lose track of what the system does, and those who write on the critical path slow the team. As AI makes code cheap, the useful contact is reading diffs, plus a little writing that is thrown away or kept off the critical path.
Gradient-boosted trees still set the baseline on tables, and the baseline is moving
12 min readA 2022 benchmark found tree ensembles ahead of deep networks on medium-sized tabular data. Tabular foundation models now claim the lead on small datasets, which changes what a sensible first experiment looks like.
A confidence score is only useful if it means what it says
12 min readA model can rank customers well and still lie about its probabilities. Calibration decides whether the number can be used in an expected-value calculation, a threshold or a handoff to a person.
A context window is not memory
12 min readAcross 200,000 simulated conversations, models averaged 39% worse when instructions arrived over several turns instead of one. Agents that run for hours need memory designed as a system.
Depth became trainable through a handful of unglamorous fixes
13 min readA network with a hundred layers multiplies a hundred numbers together, and that multiplication is where training breaks. Initialization, normalization, skip connections and optimizers each tame one part of it.
Your AI system will copy your org chart, and the eval is where the copy shows
19 min readA Microsoft study found that organizational measures predicted Windows Vista failures better than code measures did. The same logic applies to AI products, where the eval set is the artifact that falls between teams.
A model that predicts well can still be wrong about what to do
12 min readPneumonia patients with asthma looked safer in the hospital's data because they were treated more aggressively. Any model that triggers an action needs a causal question, and an experiment to answer it.
The Future of Enterprise AI: Scaling Intelligent Agents
2 min readWhy the transition from chatbots to multi-agent architectures represents the next major frontier for corporations.
An AI strategy that starts with an ambition has skipped the diagnosis
12 min readKlarna announced that an assistant did the work of 700 agents, then said cost had been too dominant a yardstick. Richard Rumelt's kernel of strategy explains why: a goal and a diagnosis are different things.
Design Principles for AI-Native Interfaces
2 min readReflecting on how user interfaces must evolve when software actively understands user intent.
Most AI pilots are designed to succeed as demos and then fail as workflows
12 min readSurveys disagree on the numbers but agree on the shape: experiments are common, scaled value is rare. The gap is organizational more than technical, and a pilot can be designed to close it.
Goodhart's law is four failures, and an ML team can budget for each one
18 min readEvery metric used to pick or reward a model decays under pressure, but the decay has four distinct mechanisms with four different fixes. The ritual most teams practice, guarding the test set, defends against the mildest of them.
Scaffolding around a model is a bet against the next model
12 min readSutton's bitter lesson is usually told about research labs. Its sharpest use is on the prompts, chains and rules teams build around a model, some of which expired within three years.
Seats, tokens or outcomes: AI broke the old pricing metrics
12 min readA seat price tracks human headcount, a flat subscription hides a cost that varies tenfold between users, and outcome pricing invites arguments about what counts as an outcome. A good metric passes four tests.
Data moats are weaker than they look, and stronger in unexpected places
12 min readLearning curves are power laws, customer value is a threshold, and foundation models have reduced what raw volume buys. The advantages that last come from outcomes, workflow position and ownership of the feedback loop.
Mixture of experts splits the price of a parameter in two
12 min readA mixture-of-experts model computes with a fraction of its weights and still has to hold all of them. That split decides what the model costs to train, to serve, and to host yourself.
Hours saved are not money earned, and AI business cases confuse the two
12 min readDevelopers who were slowed by AI believed it sped them up. Surveys report time savings that never reach a budget line. A credible business case follows hours through four conversions, each of which loses some of them.
Retrieval is an architecture decision, not a feature
13 min readAnthropic cut failed retrievals by 67% without touching the model that writes the answers. Where a RAG system gets its quality tells you which decisions are expensive to reverse.
Accuracy is the last thing a production model tells you
12 min readLabels arrive weeks or months after a prediction. Monitoring has to be built in layers, from data integrity to drift to outcomes, with alerts sized by effect and not by p-value.