DEV Community: b0gy

You built it in a weekend — now what

b0gy — Wed, 10 Jun 2026 21:05:37 +0000

No-code platforms and AI app builders have made it possible for a non-technical founder to ship a working product in a weekend. Visual app builders, spreadsheet-as-database tools, drag-and-drop workflow platforms, AI code generators. The tools are genuinely good at what they do — validating an idea with real users before writing a line of production code.

This is a net positive. More people building more things, faster. The problem starts when the validation tool becomes the production foundation. In our experience, most successful no-code apps hit platform limits within their first 18 months. Most get fully rewritten within 2 years. The tools got you here. They won't get you there.

Where platforms break

The breaking points aren't theoretical. They're specific and predictable.

No-code databases hit scaling walls earlier than most founders expect. Performance starts degrading around 50K records on visual app builders, and most platforms enforce hard row limits or API rate caps that don't budge regardless of your plan tier. When your team builds workarounds — syncing to an external database, batching requests, caching locally — you've effectively built half a custom backend anyway, just without the benefit of owning it.

Vibe-coded apps — the ones built by prompting AI coding tools — have a different failure mode. They work, but they're fragile. A Veracode study found 45% of AI-generated code contains OWASP Top 10 vulnerabilities. Test coverage in vibe-coded projects averages 12%, compared to 68% for traditionally engineered codebases. A study of 8.1 million pull requests found technical debt accumulates 30–41% faster after AI tool adoption. Georgia Tech's Vibe Security Radar has confirmed 74 CVEs directly attributable to AI-generated code, with researchers estimating the true number at 5–10x that.

The security surface is real. GitGuardian's 2026 report found 28.65 million hardcoded secrets were pushed to GitHub in 2025 — a 34% year-over-year increase. AI-assisted commits leak secrets at 2x the baseline rate. Your vibe-coded app probably has API keys in the source, no input validation on user-facing forms, and an auth implementation that looks correct but isn't.

One finding stands out. The METR randomized controlled trial — peer-reviewed, well-designed — found that experienced developers using AI coding tools were 19% slower on real tasks, despite believing they were 20% faster. The perception gap matters. If you think your AI-built app is further along than it is, you'll underinvest in the work that makes it production-ready.

The early guidance play

Here's what most founders get wrong. They treat "build on no-code" and "hire engineers" as two sequential phases with a hard cutover between them. Phase 1: validate on a no-code platform. Phase 2: rebuild everything from scratch.

That's a false binary. The decisions you make during the no-code phase determine how expensive the migration will be — or whether you need a full migration at all.

Externalize your data layer early. Use Postgres or Supabase as your database, even while running the frontend on a no-code platform. Platform-native storage is the single biggest source of migration pain. If your data already lives in a real database, you can swap the frontend without touching the data.

Use external auth. Auth0, Firebase Auth, Clerk — anything that isn't platform-native. Authentication is the hardest thing to migrate because it touches every user session. Get it right once, keep it forever.

API-first integrations. When you connect to Stripe, SendGrid, or your CRM, build the integration through APIs, not platform plugins. Plugins are convenient until you need to move. API integrations travel with you.

Document business logic separately. The rules that make your product work — pricing tiers, approval workflows, notification logic — should exist in a document, not just as visual workflows inside a no-code builder. When you eventually migrate, the engineering team needs to know what the system does, not just how it was implemented.

A fractional technical advisor at $5–15K per month during the no-code phase prevents six-figure migration costs later. The decisions that seem free now — which database, which auth provider, how integrations connect — are the ones that cost the most when you need to change them. This isn't about hiring prematurely. It's about making the no-code path viable longer and making the eventual migration cheaper.

Signs it's time to migrate

Not every no-code app needs to become a custom codebase. Some never should. But if you're seeing these signals, the clock is running.

Performance. Pages are slowing down. Automations fail silently. Users complain about load times. You've optimized everything the platform allows and it's still not enough.

Cost. Platform fees exceed 10% of revenue. Workarounds — Zapier chains, middleware services, external databases bolted on — add $500–2K per month in duct tape costs. The platform that saved you money is now the most expensive part of your stack.

Feature velocity. You're spending more time on workarounds than on new features. Every feature request starts with "can the platform even do this?" The platform is no longer accelerating you — it's the bottleneck.

Compliance. Enterprise prospects require SOC 2 or HIPAA compliance that you can't achieve on the platform. You're losing deals to a constraint you can't engineer around.

Hiring. You can't attract engineers to work on a no-code stack. The talent you need won't accept the tooling you're on. This is a leading indicator — if you can't hire for your stack, you can't maintain it.

Strategic. Investors flag platform dependency as a risk. Acquisition conversations stall during technical due diligence. Your competitive moat is gated by someone else's roadmap.

Any one of these is worth a conversation. Three or more and you're already late.

How to migrate — the strangler fig, not the big bang

The instinct is to rebuild everything from scratch. Greenfield. Clean slate. Do it right this time. This is almost always wrong.

Big bang rewrites take 2–3x longer than estimated, cost 2–3x more, and often just fail. You're maintaining two systems in parallel, shipping no new features to customers, and racing against the clock before the old system collapses or the money runs out.

The better pattern is the strangler fig — named after the tree that grows around its host. Keep shipping on no-code while you build custom for the parts that hurt most.

Migrate the most painful component first. If the platform's database is your bottleneck, migrate data to Postgres while keeping the no-code frontend. If performance on a specific workflow is killing you, rebuild that workflow as a custom API endpoint. You get immediate relief on the thing that's actually broken.

Keep no-code where it works. Internal admin tools, landing pages, simple automations, CRM integrations — these often don't need to be custom. The hybrid approach isn't a compromise. For many companies, it's the right long-term architecture.

Go custom where it hurts. Core product features, performance-sensitive flows, anything touching compliance, anything that defines your competitive advantage. These are the components that justify the engineering investment.

This approach lets you keep shipping new features throughout the migration. Your customers don't sit through a feature freeze. Your team doesn't burn out on a death march rewrite. And if priorities change midway — a new funding round, a pivot, a market shift — you can adjust without throwing away months of rebuild work.

Building the team — sequence matters

The order in which you hire is more important than who you hire.

Step 1: Fractional CTO or technical advisor. Before you hire a single engineer, get someone who can evaluate engineers, define the architecture, and set the technical direction. This person decides what to build, in what order, with what tools. Hiring engineers without this role filled is how you end up with a codebase that's worse than the no-code app it replaced.

Step 2: First 1–2 senior engineers. These aren't just your first hires — they're the people who set the engineering culture, the coding patterns, the review standards, and the testing norms. These first hires shape everything that follows. Get them right.

Step 3: Scale. Once the architecture is set and the patterns are established, bring on a mix of in-house and remote engineers. The foundation work is done. Now you can move faster.

Two mistakes we see over and over. First: promoting the first developer to CTO because they were there early. Being first isn't the same as being qualified to lead an engineering organization. Second: hiring engineers without someone who can evaluate them. If you're a non-technical founder hiring your first developer, you need a technical advisor in the room for those interviews. The downside of a bad first engineering hire is measured in months, not weeks.

What it costs

Founders ask this question first, which makes sense. But the answer depends on more than just the technology.

The technical complexity matters — a CRUD app is cheaper to rebuild than a real-time platform with complex permissions. But it's not the only driver. Here's what actually sizes a migration:

Number of users and their expectations. Migrating 500 internal users is different from migrating 50,000 paying customers. More users means more edge cases in data migration, more testing, more communication, and more risk if something breaks during cutover. Paying customers have lower tolerance for downtime than your ops team.

Number of user types and permission models. An app with one role is straightforward. An app with admins, managers, team leads, external partners, and read-only auditors — each with different data access rules — is a fundamentally different migration. Permission logic is the most tightly coupled part of a no-code app and the hardest to extract cleanly.

Integrations and data flows. Every integration is a migration boundary. Stripe billing, CRM syncs, email automation, third-party APIs, webhook chains — each one needs to be reconnected and tested in the new system. The app with 3 integrations and the app with 30 aren't the same project.

Data volume and complexity. 10K rows in a clean table migrates in an afternoon. 500K rows with nested relationships, file attachments hosted on platform CDNs, and implicit data structures baked into visual workflows — that's weeks of data engineering. And you can't get it wrong, because this is your customer data.

Compliance and regulatory requirements. If you need SOC 2, HIPAA, or GDPR compliance on the other side, the migration isn't just a rebuild — it's a rebuild with audit trails, access controls, encryption requirements, and documentation that satisfies an auditor. This adds cost and time.

Business continuity during migration. Can you afford downtime? A feature freeze? If your product needs to keep running and shipping new features throughout the migration — and it usually does — that's a parallel operation that costs more than a clean cutover.

Rough ranges based on what we've seen:

Small (few hundred users, simple permissions, limited integrations): $15–50K, 4–8 weeks
Mid (thousands of users, multiple roles, moderate integrations): $75–250K, 3–6 months
Large (tens of thousands of users, complex permissions, heavy integrations, compliance): $250K–2M+, 6–18 months

Budget 50–60% of the initial build cost annually for maintenance, infrastructure, and iteration. This isn't optional — it's the cost of running software.

Frame this as an infrastructure investment, not an expense. What revenue are platform constraints blocking today? What deals are you losing to compliance gaps? What's the cost per customer of the performance issues? The migration pays for itself when the unit economics improve — and they usually do, because the constraints you're escaping are the same ones capping your growth.

The heuristic

The cheapest migration is the one you planned for before you needed it. Spend 10% of your no-code budget on early technical guidance — externalizing data, auth, and integrations — and you'll cut 50–70% off the eventual migration cost. The decisions that feel free today are the ones with the highest interest rate.

tl;dr

The pattern. Founders build on no-code or AI tools, validate successfully, then face a six-figure rewrite because every architectural decision was locked inside the platform.
The fix. Get early technical guidance to externalize data, auth, and integrations while you're still on no-code — then migrate incrementally using the strangler fig pattern, not a big bang rewrite.
The outcome. The no-code phase lasts longer, the migration costs less, and you build an engineering team on a foundation that compounds instead of one that needs to be replaced.

Originally published at b0gy.com

AI vendor selection is not software procurement

b0gy — Mon, 01 Jun 2026 02:42:49 +0000

When you buy a SaaS tool, your data sits in someone else's database. When you buy an AI service, your data flows through someone else's model. It has the posibiliy to become training data. Your data might get embedded in weights you can never inspect. It might surface — partially reconstructed — in another customer's retrieval results.

Procurement teams treat these as the same purchase. Same checklist — SOC 2, uptime SLA, data encryption at rest — check every box, sign the contract, and discover six months later that their customer data has been training a global model by default. Or that their embeddings are locked in a proprietary format with no export path. Or that the vendor quietly added an AI sub-processor whose data handling invalidates three of their customer contracts.

SOC 2 Type II and ISO 27001 are table stakes — almost every SaaS vendor has them nowadays. They tell you the vendor has a security program. They tell you nothing about what happens to your data once it enters the model. ISO 42001 — the AI management systems standard — is the first certification that actually covers AI governance: how training data is sourced, how models are monitored, how risks like bias and hallucination are managed. Most vendors don't have it yet. The ones that do are telling you they take AI-specific risk seriously, not just infrastructure security.

Your data has more owners than you think

The thing that trips up most procurement teams is sub-processing — when your vendor passes your data to another company for processing. In traditional SaaS, this is usually a hosting provider or an analytics tool. In AI, it's the model provider. Different thing entirely.

Best example: AWS Bedrock and Claude on Anthropic's platform both let you use Claude. Both run on AWS infrastructure. They are not the same product.

AWS Bedrock: AWS is the sole processor. Anthropic has zero access to your data. Your prompts, responses, and any fine-tuning data never leave the AWS boundary. This is FedRAMP High eligible. Your existing AWS BAA covers it.

Claude Platform on AWS: Anthropic is the processor. AWS handles billing and identity. Your data flows through Anthropic's systems, governed by Anthropic's data policies. Same model. Same cloud. Completely different compliance posture.

The same pattern is everywhere. Azure OpenAI keeps data in the Azure boundary — OpenAI the company never sees it. But Azure's SRE Agent uses Anthropic as a sub-processor, and that data may leave the EU Data Boundary entirely. Salesforce lists OpenAI as a sub-processor and processed over a trillion OpenAI tokens in FY2026. Their Einstein Trust Layer claims zero data retention, but "zero retention" and "zero access" aren't the same statement.

Why does this matter if you're a US company that doesn't care about GDPR? Your customer contracts probably include a clause like "no unauthorized sub-processing." When your vendor adds an AI sub-processor you didn't know about, you may be breaching your customer contracts. Under HIPAA, your BAA with the vendor doesn't automatically cover AI sub-processors — PHI might flow through 5 or 6 sub-processors without coverage. Under ITAR, AI infrastructure with foreign nationals having admin access constitutes a deemed export — civil penalties up to $1M per violation. Under SOX, audit trail opacity in AI-processed financial data is a finding waiting to happen.

The evaluation framework

We have provided Five dimensions. For each: what to ask, what good looks like, and the red flag. These are the things your SaaS checklist skips or gets wrong.

None of this matters if the tool doesn't solve a real problem. Technical due diligence isn't a substitute for asking: does this vendor actually help us move faster, serve customers better, or cut cost? The framework below assumes you've already established business value. What it protects is your ability to keep that value — by making sure the vendor's data practices, lock-in posture, and contractual commitments don't turn a good tool into an expensive liability.

1. Data usage and training defaults

This is the one that burns people. The question isn't "is my data secure?" It's "is my data making the vendor's product better for everyone else?"

The defaults are worse than you'd expect. ChatGPT trains on user data by default — even on paid plans. Team and Enterprise tiers don't, but Plus does. GitHub Copilot, as of April 2026, uses interaction data from Free, Pro, and Pro+ plans for training by default. Salesforce's Spring '26 release feeds customer data into global predictive AI models, on by default. Atlassian will use customer metadata to train Rovo starting August 2026 — and Free, Standard, and Premium tiers can't opt out of metadata collection.

What to ask: "Show me the written policy on data usage for model training — not the marketing page, the contractual language. Does it distinguish between content, metadata, and interaction patterns? Is opt-out a settings toggle or a contractual guarantee?"

What good looks like: A contractual commitment — not a toggle — that customer data is not used for training. A clear definition of what "data" means — including metadata and usage patterns, not just content. A specific policy on what happens to data after inference.

Red flag: "We don't train on your data" but the ToS says otherwise. Zoom updated their terms in 2023 to grant a perpetual license to customer content for ML/AI — the backlash forced three revisions in a week. Slack opts customers in for non-generative AI training on messages by default, and opting out requires contacting support. If the vendor's answer to "do you train on my data" requires a footnote, the answer is yes.

The gate: Know exactly what data gets used, for what purpose, and whether that's acceptable for your use case. Some services are designed to learn from your data — that might be the value proposition. The question isn't "does it train" but "do you know what it trains on, did you consent, and can you control it?" If the answer to any of those is no, that's the problem.

2. Sub-processor transparency

Most vendors will hand you their SOC 2 and/or ISO 27001 report unprompted. Ask for their sub-processor list and watch the conversation shift.

What to ask: "Provide a complete list of sub-processors that handle customer data, including AI model providers. What data does each sub-processor receive? What is your notification process when you add a new AI sub-processor?"

What good looks like: A published sub-processor list — updated regularly — with 30+ day advance notice before adding new ones and contractual objection rights. Specific disclosure of which data each sub-processor receives. Nightfall AI maintains a public AI sub-processor tracker for major SaaS vendors — if your vendor isn't on it, ask why.

Red flag: No sub-processor list. Or a list that omits AI model providers. Or notification "at our discretion." If the vendor can't tell you exactly who touches your data, they either don't know or don't want you to.

The gate: You need to know who touches your data. How formal that looks — published list, advance notice, objection rights — depends on your regulatory environment and customer commitments. But the baseline is: the vendor can tell you, clearly, who their sub-processors are and what data each one receives.

3. Data security and incident response

Samsung employees pasted proprietary source code into ChatGPT 3 times in 20 days in 2023. Samsung banned all generative AI. In April 2026, an employee at Vercel with OAuth delegation access to Context AI's systems was compromised — the breach data ended up on BreachForums, priced at $2M. An unauthenticated endpoint in AnythingLLM's Pinecone integration exposed API keys. A supply chain attack hid malicious instructions in GitHub READMEs that executed when developers used Cursor IDE's AI features.

This is the attack surface now. It isn't shrinking.

What to ask: "What's your incident notification timeline? Do you have AI-specific indemnity provisions — if your model produces infringing output that ends up in my product, who carries liability? What controls exist for agent execution, and can I kill them remotely?"

What good looks like: 72-hour incident notification. AI-specific indemnity for IP infringement on outputs. A kill switch for agent execution. Explicit acknowledgment that prompts are your IP.

Red flag: Boilerplate SaaS security language with no AI-specific provisions. If their incident response plan doesn't mention model-specific attack vectors — prompt injection, embedding inversion, training data extraction — it was written before they added AI to the product.

4. Lock-in and portability

AI lock-in runs deeper than traditional software lock-in. With SaaS, your data is in their database and you need an export. With an AI vendor, your data might be in their embeddings (proprietary format, no export), in a fine-tuned model on their infrastructure (weights not exportable), and woven into an orchestration layer that couples your workflows to their tooling. Three layers where traditional software has one.

Every month of investment increases exit cost — and exit cost is a business risk, not just a technical one. If a better model ships next quarter and you can't switch, you're paying more for less. If your vendor gets acquired and the roadmap shifts, you're stuck. Re-embedding 10 million documents costs $30–$900 in API fees alone, plus the engineering time to validate that new embeddings produce equivalent retrieval quality. Fine-tuned model weights generally aren't exportable from managed services. Workflow logic built on vendor-specific orchestration APIs requires a rewrite, not a migration.

What to ask: "Can I export my embeddings, fine-tuning datasets, prompts, and interaction logs in open formats — JSON, CSV, Parquet? What is your model deprecation policy — how much notice, and do I retain access to the previous version during the notice period? What happens to my data if you're acquired or shut down?"

What good looks like: Contractual data exit provisions covering embeddings, fine-tuned model weights, and workflow configurations. 90-day minimum deprecation notice with continued access to the previous version. Data export in open formats, not proprietary dumps.

Red flag: "We'll work with you on migration." That's not a contractual guarantee — it's a vibe. Notion AI is instructive here: Enterprise plans get zero retention with LLM providers. Non-Enterprise plans allow up to 30 days of retention. Same product, different data posture based on the plan you're on.

5. Jurisdictional sovereignty

The legal ground under AI data transfers is moving faster than most legal teams can track. The EU-US Data Privacy Framework was invalidated in late 2025. CNIL issued guidance in February 2026 requiring supplementary measures beyond Standard Contractual Clauses — including encryption where the provider doesn't hold the keys. That's technically impossible for most AI API usage. The model needs plaintext to process it.

Even if you're US-only, the regulatory surface is expanding. Colorado's AI Act takes effect June 30, 2026. Texas RAIGA hit January 1, 2026. Illinois followed in February 2026. The US CLOUD Act means US-headquartered providers are subject to extraterritorial data access regardless of where servers sit — which makes "data residency in the EU" a weaker guarantee than it sounds.

What to ask: "Where is my data processed, stored, and in transit? Which jurisdictions does it touch? What happens when a government requests access to my data under CLOUD Act or equivalent?"

What good looks like: Specific answers — not "it depends." A vendor who can name the regions, the legal mechanisms, and the fallback plan. 61% of Western European CIOs now prioritize local providers. That number wasn't close 2 years ago.

Red flag: "Our servers are in the US/EU." That's not an answer. Server location tells you where data sleeps, not where it travels.

Deployment model changes the risk, not the framework

Everything above assumes cloud-hosted SaaS. But AI gets deployed on-prem, hybrid, and air-gapped too — and the risk profile shifts depending on where the model runs.

Cloud / SaaS. All five dimensions apply at full weight. Sub-processor risk is highest here — your data leaves your boundary by design. Training defaults matter most because you have the least control over what the vendor does with data after inference. This is where most enterprises start and where most procurement failures happen.

Hybrid. Inference might run on your infrastructure, but model updates, fine-tuning, or telemetry may still call home. The sub-processor question doesn't disappear — it just gets harder to answer. Ask specifically: what data leaves my environment, when, and to where? A vendor who says "nothing leaves" but ships telemetry to their cloud for model monitoring isn't being straight with you. Sovereignty risk doesn't go away either — data in transit between your on-prem components and the vendor's cloud still crosses jurisdictional boundaries. If your cloud component runs in us-east-1 but your on-prem sits in Frankfurt, you've got a transatlantic data flow that CNIL's 2026 guidance doesn't smile on. Lock-in risk is higher here because you've invested in on-prem infrastructure that's coupled to the vendor's model format and orchestration.

On-prem. Training defaults and sub-processor risk drop — but don't assume they disappear. Many on-prem AI deployments still phone home: license validation, telemetry, usage metering, model update checks, and crash reporting all create outbound data flows you probably haven't audited. Ask the vendor for a complete list of outbound connections the software makes and what data each one carries. If it genuinely doesn't phone home, treat it as air-gapped and enforce network rules accordingly — don't leave outbound access open on trust. Lock-in and portability become the dominant risks. You're running their model on your hardware, likely with their orchestration tooling, and if they deprecate the model version or change licensing terms, you're exposed. AI governance shifts to you too — bias testing, output monitoring, and model lifecycle management are now your ops problem, not the vendor's.

Air-gapped. Sovereignty and sub-processor concerns are mostly solved by architecture. What remains is lock-in (you're deeply coupled to whatever you deployed), model currency (you can't easily update), and operational governance (you own the entire AI risk surface with no vendor support). The contract conversation shifts from "what do you do with my data" to "what happens when I need a model update and can't connect to your systems."

The five evaluation dimensions still apply in every case — but the weight shifts. Use the scoring table below and adjust weights based on your deployment model. A cloud deployment should weight security, data handling, and sub-processors highest. An air-gapped deployment should weight lock-in, portability, and governance highest.

How to score

Keep the rubric to 15-20 criteria across these five dimensions. Anything more causes evaluation fatigue and everything gets scored a 3. Open the scorecard (Google Sheet).

Baseline checks first. Before you score, establish where the vendor stands on the fundamentals:

Security certifications — SOC 2 Type II and ISO 27001 are standard. If they don't have them, ask why — but a smaller vendor solving a real problem might not have them yet. That's a risk to manage, not necessarily a dealbreaker.
Data usage transparency — you understand what data is used for training, whether you consented, and whether you can control it. The right answer depends on your use case — but "we don't know" is never the right answer.
Sub-processor visibility — the vendor can tell you who touches your data and what each party receives. The formality of disclosure (published list, advance notice, objection rights) scales with your regulatory exposure and customer commitments.

If a vendor can't clear these basics, proceed with caution. But "proceed with caution" isn't "walk away" — the business need, the alternatives, and the risk you can actually manage all factor in.

Then score on a 1–5 scale across these categories. Weight them based on what matters most to your organization — a healthcare company will weight data handling differently than a startup building internal tools.

Category	What it covers
Security & privacy	Incident response, access controls, AI-specific security
Data handling	Training defaults, retention, deletion, sub-processors
AI governance	Model transparency, bias testing, output monitoring, ISO 42001
Technical fit	Model quality for your use case, latency, reliability
Integration & portability	API design, export capabilities, lock-in mitigation
Business & vendor viability	Vendor financial health, TCO, support, business continuity, roadmap alignment

Who needs to be in the room. Not just procurement. You need CISO or security lead, privacy/DPO, legal, engineering, and the business unit owner. The business owner isn't there to rubber-stamp — they're there to validate the vendor solves a real problem, that total cost of ownership (not just per-token price) makes sense, and that the vendor's roadmap aligns with where the business is headed. 80% of enterprise AI purchases now face stricter scrutiny than standard SaaS — if your process doesn't reflect that, you're rubber-stamping risk.

Timeline. Plan for 3-6 months, not the 4-8 weeks of standard SaaS procurement. The extra time isn't bureaucracy — it's what it takes to evaluate dimensions that don't exist in your SaaS checklist. Use part of that time to run a proof of concept against your actual use case. A vendor who scores well on paper but underperforms on your data, your latency requirements, or your team's workflow is still the wrong vendor.

Contract provisions worth negotiating

Six clauses that aren't standard in SaaS agreements but matter for AI. Not every vendor will agree to all of them, and not every deal requires all of them. The point is knowing what you're accepting when you sign without them.

Data usage for training — understand and document what's used, what's not, and who controls it. If training on your data is the product, make sure the terms reflect that explicitly — not buried in a ToS update.
Sub-processor disclosure — know who touches your data. The formality (published list, advance notice, objection rights) should match your regulatory exposure.
Model deprecation — understand the vendor's deprecation policy. How much notice? Can you pin to a version during transition? What's the migration path?
Output IP — clarify ownership of outputs and whether the vendor acknowledges prompts as your IP. This matters more in some industries than others.
Data deletion — understand whether deletion extends to embeddings, vectors, and sub-processors. Ask for certification of completion if your compliance framework requires it.
Export — can you get your prompts, logs, embeddings, and fine-tuning datasets out in open formats? The answer shapes your exit cost.

If a vendor won't put their marketing claims in the contract, that tells you something. It might be a dealbreaker — or it might be a risk you accept because the business value justifies it. Either way, make that call consciously, not by default.

The heuristic

Add one question to every AI vendor evaluation that your SaaS checklist never asks: "Show me every company that touches my data after I hit send, and show me the contract that governs each one." The vendor who answers clearly is worth signing. The vendor who hedges is the one who will surprise you.

tl;dr

The pattern. Teams evaluate AI vendors with their SaaS procurement rubric — checking SOC 2 and uptime SLAs — while missing the AI-specific risks that actually burn them: default training on customer data, undisclosed sub-processors, proprietary lock-in, and sovereignty gaps.
The fix. Validate business value first — then run AI vendors through the dimensions SaaS checklists skip: training defaults, sub-processor transparency, AI-specific security, portability, vendor viability, and jurisdiction.
The outcome. You pick vendors that solve real business problems and protect your data at the model layer, not just the platform layer. Your exit cost stays manageable because you negotiated portability before you had no leverage.

Originally published at b0gy.com

Hiring engineers in the age of AI

b0gy — Sun, 24 May 2026 02:39:38 +0000

Your engineering interview loop was designed for a world where the bottleneck was writing code. The best engineers in 2026 are not the fastest coders — they're the ones with the best judgment about what to build, how to direct AI toward the right outcome, and when to override what it produced.

The job title still says "engineer," but the role has fractured. The person you actually need is part architect, part product thinker, part operator, and part team multiplier — designs systems before touching a keyboard, kills bad ideas before they become features, plans for the 3am page before it fires, makes the people around them better. AI writes the code. The human decides whether the code should exist at all.

Augment Code put it well in their hiring framework: the human role has shifted from author to architect and editor. You define intent, make trade-off decisions, set guardrails, and serve as the last line of quality. Raw coding ability no longer separates exceptional engineers from competent ones — regardless of whether they're writing backends, frontends, data pipelines, or infrastructure.

CoderPad's 2026 State of Tech Hiring confirms this from the demand side — technical assessments are up 48% globally, and 82% of developers find AI at least somewhat useful. Companies leading in AI are hiring more engineers, not fewer. The bottleneck isn't code generation. It's judgment, systems thinking, and the ability to work alongside AI without losing control of the outcome.

If your interview loop still centers on "can this person write a balanced binary tree on a whiteboard," you're selecting for a skill that AI does better than any human. Here's what to select for instead.

The rubric

Six dimensions, each with sub-criteria you score independently. 3 = strong, 2 = adequate, 1 = weak. Open the scorecard (Google Sheet).

1. Architecture & design

Can the candidate think at the level of systems, not just functions? These apply whether someone is building APIs, frontends, data pipelines, or infrastructure.

Trade-off analysis under ambiguity — can they explain why one approach beats another in terms the business cares about, not just technical preference?
Failure mode awareness — do they design for the 3am outage, or only the happy path?
Scale reasoning — do they ask about constraints (current bottlenecks, data patterns, cost) before proposing solutions?
API and contract design — can they design interfaces that other teams can integrate safely?
Data modeling and storage decisions — do they choose the right storage for the access pattern, not just the storage they know?

2. AI-assisted execution

Can the candidate direct AI tools toward the right outcome — and catch it when the output is wrong?

Directs AI toward well-defined subtasks — uses AI for implementation while keeping control of the overall design. Treats AI as a tool, not an oracle.
Critically reviews and debugs AI output — can explain why AI-generated code is or isn't correct. This is the single strongest signal.
Clear boundary between AI-delegated and human-owned work — knows which tasks to hand off and which to protect. No boundary means no judgment.
Understands AI limitations in production contexts — knows where models hallucinate, where context windows break, where AI-generated code introduces security or concurrency risks.

3. Systems & operations

Does the candidate understand what makes code production-worthy?

Observability — thinks about monitoring, alerting, and SLOs from day one. Monitors what the user cares about, not what the default dashboard provides.
Debugging methodology — has a systematic approach: logs, metrics, traces, reproduction. Asks what changed recently before guessing.
Cost awareness — considers compute, storage, and API call costs as design constraints, not afterthoughts.
Incident response and on-call mindset — has a mental model for what happens after the code is merged. Designs for operability.

4. Product & problem selection

Does the candidate solve the right problems, or just the ones assigned?

Asks about users and business context before building — wants to know the success metric, not just the spec.
Pushes back on problem framing when appropriate — would rather solve a simpler problem well than an interesting problem poorly.
Chooses simple solutions over interesting ones — the boring choice that saves the team real time or pain.

5. Learning velocity

How fast does the candidate adapt when tools and practices change underneath them?

Has adopted and discarded tools based on results — not just tried things, but evaluated and made deliberate choices.
Can articulate what changed in their workflow and why — adoption without displacement usually means the tool isn't actually being used.
Stays current without hype-chasing — runs personal experiments but doesn't chase every new release.

6. Culture & collaboration

AI amplifies individual output — but a 10x individual who can't collaborate is a net negative. Score this separately from technical ability.

Curiosity — asks questions, explores unfamiliar territory, wants to understand the system beyond their immediate scope.
Navigates disagreement constructively — describes what they learned from technical disagreements, not just how they won them.
Communicates technical decisions to non-engineers — can explain trade-offs in terms a PM or exec can act on.
Shares context on AI-generated code with the team — flags AI-generated sections in PRs, documents reasoning, takes responsibility for the review burden.
Mentors or lifts others — impact isn't just individual output. Improves the people around them.
Holds opinions loosely — has strong technical convictions but adapts to team norms and new information.

The questions

Fourteen questions, grouped by what they assess. Each one includes what you're listening for and the red flag.

Architecture & design

1. "Walk me through a system you designed that had to handle a non-obvious failure mode. What was the failure, and how did your design account for it?"

Listen for: specifics about the failure mode, the trade-off they made, and whether they designed for it proactively or reactively. Strong candidates talk about the failure before it happened.

Red flag: only discusses happy-path architecture.

2. "You're designing a service that needs to process 10x its current load within 6 months. Where do you start?"

Listen for: questions before answers. Strong candidates ask about current bottlenecks, data access patterns, and cost constraints before proposing solutions. They think about what not to change as much as what to change.

Red flag: immediately proposes a technology ("just use Kafka") without understanding the constraints.

3. "Tell me about a time you chose a boring solution over an interesting one. Why?"

Listen for: a clear trade-off between technical elegance and operational simplicity. The best answer is a story where the boring choice saved the team real time or pain.

Red flag: can't think of one. Engineers who always choose the interesting solution are expensive to operate alongside.

AI-assisted execution

4. "Walk me through your daily workflow. Where does AI fit in, and where doesn't it?"

Listen for: specificity. Strong candidates can name exact tools, describe which tasks they delegate to AI, and — critically — which tasks they don't. The boundary matters more than the tools.

Red flag: vague answers like "I use Copilot for everything." No boundary means no judgment.

5. "Tell me about a time AI-generated code was wrong in a way that wasn't obvious. How did you catch it?"

Listen for: a real story with a real failure mode. Strong candidates describe a subtle bug — a race condition, a missed edge case, a security hole — that they caught through review, testing, or domain knowledge. This is the single strongest signal for AI-assisted work: the ability to be the quality backstop.

Red flag: "that hasn't really happened to me." It has. They just didn't notice.

6. "If I gave you a new codebase you've never seen and asked you to add a feature using AI tools, how would you approach it?"

Listen for: a process that starts with understanding, not prompting. Strong candidates talk about reading the existing code, understanding the architecture, then using AI for implementation — not pasting requirements into a chat window and hoping.

Red flag: leads with "I'd prompt the AI to..." without any mention of understanding the system first.

Systems & operations

7. "Your service is throwing 500s for 2% of requests. Walk me through your debugging process."

Listen for: a systematic approach — logs, metrics, traces, reproduction. Strong candidates ask what changed recently, check deployment history, and think about partial failures. Let them use AI tools during the discussion. Watch whether they use AI to accelerate diagnosis or to replace thinking.

Red flag: guesses without data. "It's probably the database" is not a debugging process.

8. "How do you decide what to monitor in a new service?"

Listen for: a framework, not a checklist. Strong candidates think about SLOs, user-facing metrics, and leading indicators of failure — not just CPU and memory. They monitor the thing the user cares about, not the thing the infrastructure provides by default.

Red flag: "whatever the default dashboard gives us."

Product & problem selection

9. "Tell me about a feature you decided not to build. What was the reasoning?"

Listen for: business judgment. The feature was technically feasible but wrong for the user, the timeline, or the system's current maturity. Strong candidates kill their own ideas.

Red flag: can't think of a feature they chose not to build. This suggests they build whatever's asked without filtering.

10. "A PM asks you to add a feature that you think is a bad idea. What do you do?"

Listen for: pushback with evidence, not ego. Strong candidates describe how they'd frame the concern — data, user impact, operational cost — and what they'd do if overruled. The answer to "what if they still want it" matters as much as the initial pushback.

Red flag: "I'd just build it, they're the PM." Or the opposite: "I'd refuse." Neither is a real answer.

Learning velocity

11. "What AI tool or workflow did you adopt in the last 6 months that meaningfully changed how you work? What did it replace?"

Listen for: a concrete change in behavior, not a tool name. Strong candidates describe what they stopped doing when they adopted the new thing. Adoption without displacement usually means the tool isn't actually being used.

Red flag: names a tool but can't describe the workflow change. "I started using Cursor" is not an answer. "I stopped writing boilerplate tests by hand because Cursor generates them and I review them" is.

12. "What's something AI tools are bad at today that people assume they're good at?"

Listen for: nuance and firsthand experience. Strong candidates have bumped into real limitations — context window issues, hallucinated APIs, broken concurrency patterns — and can describe them specifically. This question reveals whether someone has used AI tools enough to know where they break.

Red flag: "AI is pretty good at everything now." It is not.

Culture & collaboration

13. "Tell me about a technical disagreement you had with a teammate. How did it resolve, and would you do anything differently?"

Listen for: respect for the other person's view, willingness to change their mind, and a resolution that wasn't just "I was right." Strong candidates describe what they learned, not just how they won.

Red flag: every story ends with them being right. Or they can't recall a disagreement — which means they either avoid conflict or don't notice it.

14. "How do you share context with your team when you've used AI to generate a significant chunk of code?"

Listen for: a process — PR descriptions that call out AI-generated sections, documentation of the reasoning, flagging areas that need extra review. Strong candidates recognize that AI-generated code shifts the review burden to the team and own that.

Red flag: "I just push it like any other code." That's how AI-generated bugs become the team's problem.

Red flags to watch across the loop

A few patterns that span the whole loop:

Tool fluency without technical depth. The candidate is fast with AI tools but can't explain the code they produce. This is the most dangerous hire in 2026 — someone who ships faster but embeds hidden defects. Canva's interview team found the same thing: the best candidates don't just prompt — they ask clarifying questions, use AI for well-defined subtasks, and critically review output.

No boundary between AI tasks and human tasks. Every engineer needs a clear mental model of what they delegate and what they don't. If someone uses AI for everything indiscriminately, they've abdicated judgment.

Resistance to AI as identity. Some candidates treat not using AI as a badge. In 2026, this is like refusing to use an IDE. It doesn't demonstrate skill — it demonstrates rigidity.

Can't debug AI output. If you ask them to review AI-generated code and they can't identify issues, they're a liability in a codebase where AI writes the first draft.

The heuristic

You're not hiring a coder anymore. The engineer you need in 2026 is an architect who designs systems before writing them, a product thinker who kills bad ideas before they become tickets, an operator who plans for the 3am page, and a teammate who makes everyone around them better. AI handles the typing. You're hiring for everything else. Interview for judgment, not keystrokes — the candidate who pushes back on AI output is worth more than the one who prompts their way to a solution.

tl;dr

The pattern. Engineering interviews still optimize for raw coding ability — a skill AI now handles — while ignoring the architect, product thinker, operator, and collaborator the role actually demands.
The fix. Score candidates across six dimensions (architecture, AI-assisted execution, systems & ops, product taste, learning velocity, culture) with specific sub-criteria, and use questions that reveal how they direct, evaluate, and override AI output.
The outcome. You hire engineers who think like architects, challenge like PMs, operate like SREs, and use AI as a tool — not engineers who either ignore it or depend on it blindly.

Originally published at b0gy.com

The displacement gap

b0gy — Sat, 16 May 2026 16:43:44 +0000

Every few weeks a new headline lands: 85 million jobs displaced. 300 million full-time roles affected. All white-collar work automated within 18 months. The numbers are big, round, and — if you look closely — mostly about theoretical capability, not observed reality.

Anthropic published labor market research in March that introduced a distinction most coverage ignores: the gap between what AI can automate and what it is automating. That gap is enormous. And it tells a very different story than the headlines.

The gap is the story

Anthropic's key contribution is a measure they call "observed exposure" — what Claude actually does in production, as opposed to what benchmarks say it could do.

The numbers are striking. Computer and math occupations have 94% theoretical AI exposure. In practice, Claude covers 33%. Office and admin roles — the ones everyone assumes are already gone — show 90% theoretical exposure and a fraction of that in actual use.

97% of observed Claude usage falls within theoretically feasible categories. The model can do the work. Organizations just aren't deploying it that way.

This is not a technology constraint. It is an adoption constraint. The bottleneck is not whether the model can draft a financial analysis or triage a support ticket. It is whether the organization has rebuilt the workflow, retrained the team, and instrumented the process to actually use AI where it fits.

Mustafa Suleyman predicted in February that all white-collar tasks would be automated within 12-18 months. Anthropic's data says we're at a third of theoretical capacity today, with no clear acceleration in the adoption curve. Both things can be true — the capability is there, the deployment is not — but the implication is different from what the headlines suggest.

The hollowing-out is real, just quiet

No clear evidence that AI has increased overall unemployment. That's the headline finding, and it's technically correct. But underneath it, something more specific is happening.

Anthropic found a 14% drop in the job-finding rate for workers aged 22-25 in AI-exposed occupations since ChatGPT launched. Not mass layoffs — reduced hiring. The entry-level pipeline is narrowing. Companies are not firing customer service reps. They are not backfilling them when they leave.

This is the pattern that matters for anyone running an AI program. The displacement is not dramatic. It is a slow compression of roles at the bottom of the org chart — data entry, basic admin, junior financial analysis, first-tier customer support. The people in these roles are not losing their jobs tomorrow. They are losing the next version of their jobs — the promotion, the adjacent role, the career ladder that used to exist.

US data suggests roughly 25,000 jobs erased per month against 9,000 new ones created. A net loss of 16,000 per month sounds alarming until you put it against a labor force of 160 million. But zoom in on who's affected and the picture sharpens: entry-level workers, administrative roles, and — disproportionately — women.

The gender gap nobody is planning for

79% of employed US women work in roles at high risk of automation, compared to 58% of men. Of the 6 million US workers most directly exposed to AI displacement, more than 85% are women. The ILO found women more exposed than men in 88% of countries analyzed.

This is not a coincidence. Women are overrepresented in exactly the categories AI automates first: administrative support, clerical work, payroll, reception, data entry. These roles have high theoretical exposure and high observed exposure — the gap between capability and deployment is smaller here because the tasks are more structured, more repetitive, and easier to automate without rebuilding an entire workflow.

Harvard and Berkeley researchers found a 25% gap in AI adoption between men and women. The workers most exposed to displacement are also the least likely to be building fluency with the tools that could reshape their roles instead of eliminating them.

Most workforce planning we see treats AI displacement as role-neutral. It is not. If your reskilling program does not specifically account for which roles and which demographics are most exposed, you are planning for a workforce transition that does not match the actual transition happening.

Reskilling is not a slide in the deck

IBM estimates 40% of the global workforce needs new skills within three years. The WEF projects 92 million jobs disappearing by 2030, offset by 170 million new roles — but those new roles require different skills, and the people losing the old roles are not automatically qualified for the new ones.

The new jobs are real. AI trainers, explainability engineers, data annotators, forward-deployed AI engineers — 1.3 million new roles by various estimates, plus 600,000 data center positions. Workers with advanced AI skills earn 56% more than peers. The demand side is genuine.

The problem is the bridge. BCG and HBR frame it well: AI will reshape more jobs than it replaces. 50-55% of US jobs will be substantially changed, not eliminated. But "substantially changed" means the person in the role needs to learn new tools, adopt new workflows, and develop judgment about when to trust AI output and when to override it. That is a training problem. And 90% of global enterprises report critical skills shortages going into 2026.

We see this pattern in every engagement. The organization has a model in production. The AI team is shipping features. But nobody owns the question of how the people whose workflows just changed are supposed to adapt. There is no training program. There is no measurement of whether adoption is happening at the individual level. There is a Slack message that says "we now have an AI tool for X" and an expectation that people will figure it out.

They don't.

Three phases, one window

The research broadly converges on a phased timeline. 2023-2025 was task automation, hiring freezes, role compression. 2026-2028 — where we are now — is when career transition spikes and displacement peaks. 2028 onward is the new equilibrium, where the job market has restructured around AI-augmented roles.

If that timeline is roughly right, organizations have about two years to get serious about the workforce side of their AI programs. Not the model side. Not the infrastructure side. The people side.

That means identifying which roles are being compressed — not theoretically, but based on actual usage data. It means building reskilling programs targeted at the specific demographics and job families most exposed. It means measuring adoption at the individual level, not the org level, because a 60% adoption rate can mean 60% of the team uses AI daily or 100% of the team opened the tool once. Those are very different situations.

The heuristic

Your AI workforce problem is not that machines will replace your people. It is that you are changing what the work requires without changing how your people are prepared to do it. Close that gap before the market closes it for you.

tl;dr

The pattern. Organizations focus on what AI can automate in theory while ignoring that actual deployment is a fraction of capability — and the real displacement is a quiet hollowing-out of entry-level and administrative roles, disproportionately affecting women.
The fix. Treat reskilling as an engineering problem: identify which roles are actually changing based on usage data, build targeted training for the specific demographics most exposed, and measure adoption at the individual level.
The outcome. Your workforce adapts alongside your AI program instead of being hollowed out by it, and the people most at risk become the people most prepared for what comes next.

Originally published at b0gy.com

The AI divide is not about AI

b0gy — Sun, 10 May 2026 22:12:17 +0000

PwC's 2026 AI Performance Study dropped a number that should worry anyone running an AI program: three-quarters of the economic value from AI is being captured by 20% of companies. The other 80% are spending real money and getting marginal returns.

This isn't a technology problem. It's an organizational one.

The numbers are worse than they look

Stanford's 2026 AI Index puts organizational adoption at 88%. Nearly every company of meaningful size is doing something with AI. But doing something isn't the same as getting value from it.

Writer's enterprise survey of 2,400 knowledge workers found that 79% of organizations face challenges in adoption — a double-digit increase from 2025. Nearly half of leaders say AI adoption has been a "massive disappointment." And 54% of C-suite executives say it's tearing their company apart.

Meanwhile, the top 20% are pulling away. PwC found those companies are 2.6x more likely to say AI is reshaping their business model, not just trimming costs. They're using AI for growth, not efficiency.

The gap is accelerating.

Efficiency is the wrong goal

Most companies failing at AI are doing exactly what the consultants told them to do: find inefficiencies, apply AI, measure cost savings. The problem is that efficiency gains from AI are real but small. Stanford documents 14-15% productivity improvements in customer support, 26% in software development. Meaningful — but not transformative.

The companies capturing most of the value are doing something different. They're using AI to enter adjacent markets, create new product categories, and restructure how their business works. PwC's single strongest predictor of AI-driven financial performance wasn't model quality or engineering talent — it was the ability to pursue growth opportunities from industry convergence.

The winners aren't using AI to do the same things cheaper. They're using it to do different things entirely.

The strategy problem is real

Here's the part that should alarm leadership teams. Writer found that 39% of companies investing over a million dollars annually in AI don't have a formal strategy for generating revenue from it. Among those that do, 75% of executives admit the strategy is "more for show than for actual internal guidance."

Three-quarters of AI strategies are slide decks that no one follows.

We see this constantly. A company has a model in production, a team maintaining it, a budget approved — and no clear answer to "what is this doing for the business?" The AI team ships features. Leadership counts deployments. Nobody measures whether the deployment changed anything that mattered.

The people divide is the real divide

The most uncomfortable finding in the Writer survey: 92% of C-suite executives are cultivating a new class of "AI elite" employees. 60% plan to lay off those who won't adopt AI. AI super-users are 3x more likely to get a raise or promotion and 5x more productive than slow adopters.

This creates a two-tier workforce inside companies that already have a two-tier AI strategy. The people who are good at using AI tools get more resources, more visibility, and more autonomy. Everyone else gets a mandate to "adopt AI" with no clarity on what that means.

The organizations closing the divide are the ones being explicit about what AI adoption looks like at each role level — not "use AI" but "here are the three workflows that change, here is the training, here is how we measure whether it worked."

What the top 20% actually do differently

After working with organizations on both sides of this divide, the pattern is consistent:

They fund AI at the product level, not the infrastructure level. The lagging companies build platforms and wait for use cases. The leading companies start with a business outcome and work backward to what needs to change.

They measure business metrics, not AI metrics. Not F1 scores or latency percentiles — revenue per user, time to close, customer retention. If the AI team can't connect their work to a metric the CFO cares about, they rethink the work.

They treat AI as a management problem. The technology is commoditized. GPT-5.5, Claude Opus 4.7, and DeepSeek V4 all shipped within days of each other in April. Model quality is converging. The differentiator is how the organization integrates, governs, and iterates on AI-driven processes.

They accept that most experiments will fail. The lagging companies run one pilot, see mediocre results, and declare AI overhyped. The leading companies run twenty pilots, kill fifteen, and scale the five that work. The failure rate is the same. The response to failure is different.

The heuristic

If your organization is spending seven figures on AI and you can't clearly articulate what business outcome has changed as a result — you're in the 80%.

That's not a reason to stop. It's a reason to stop doing what you're doing and start treating AI as a business decision rather than a technology project. The divide isn't about who has the best models. It's about who has the clearest thinking about what those models are for.

tl;dr

The pattern. 80% of companies are spending real money on AI and getting marginal returns because they treat it as a technology project — find inefficiencies, apply AI, measure cost savings.
The fix. Fund AI at the product level with business metrics, run multiple bets with kill criteria, and define what "AI adoption" means concretely for each role instead of issuing mandates.
The outcome. AI drives growth — new markets, new products, new business models — instead of shaving single-digit percentages off existing processes.

Originally published at b0gy.com