[Power and Prediction] Chapter 2: The Between Times

~ pdawg ~ Leave a comment

If AI is so good at prediction, where are the results? That’s the uncomfortable question Chapter 2 opens with. Over the past decade, measured productivity growth has fallen by roughly half, and for a majority of Americans real income has barely moved since the late 1990s. Machines can now read X-rays, translate languages, and drive cars, yet the aggregate numbers look like nothing much is happening.

The authors are quick to point out that this productivity paradox is not new. In the 1980s, computers were spreading through every office while the productivity statistics stayed flat, prompting economist Robert Solow’s famous observation that the computer age was visible everywhere except in the numbers. The gains arrived eventually, but only after businesses stopped using computers to speed up old processes and started redesigning the processes themselves.

The book gives this stretch a name: the Between Times. It’s the period after we’ve seen what a technology can do and before it has been widely adopted in ways that change how things actually work. We are in AI’s Between Times now. Point solutions and application solutions can be designed and deployed quickly, which is why we see so many of them. System solutions, the ones that will unlock AI’s real potential, take far longer.

Three kinds of solutions, and what actually separates them. The book’s distinction between point, application, and system solutions is the spine of the whole argument, so it’s worth making it concrete.

A point solution takes a decision your organization already makes and makes it better, without touching anything else. The decision was there before the AI arrived; the AI just improves the prediction feeding it. A bank already decides which transactions to flag as fraud; swap the old rules engine for a machine-learning model and you catch more fraud with fewer false alarms. A retailer already forecasts demand; replace the analyst’s spreadsheet with a model and the forecast gets sharper. Nothing downstream has to change. Same people, same workflow, same approvals, just a better input. This is the AI equivalent of bolting an electric motor onto the old central shaft.

An application solution uses prediction to create a decision that didn’t exist before, but still slots into the system as it stands. Nobody at a record label used to decide, song by song, what each listener should hear next. Streaming recommendations made that a decision, and a valuable one. A thermostat that learns when you’re home and adjusts itself is making a decision no human was making. These are new capabilities, not upgrades, but they’re self-contained. You can add them to the existing business without rethinking how the rest of it operates. This is the individual electric drive on each machine: new and useful, but the factory is still built the old way.

A system solution is what happens when a better prediction changes what everyone else should be doing. Consider a hospital that deploys an AI to predict which emergency patients will deteriorate. As a point solution, it’s a flag on a screen that a nurse may or may not act on. But if the prediction is reliable, the right response is to change how triage works, how beds are allocated, how staff are scheduled, and which patients get admitted at all. The value of the prediction isn’t in the flag; it’s in the reorganized hospital. Or consider retail: if you can predict demand well enough, you might ship products toward customers before they order, which upends inventory, logistics, returns, and pricing all at once. The prediction is the trigger, but the payoff lives in everything that has to be rebuilt around it. This is the single-story factory designed around the flow of work.

The test is independence. The authors boil the distinction down to one question: does the value of the improved decision stand on its own, or does it depend on other decisions changing too? A useful way to check is to imagine switching the AI off. If the only thing that happens is that one decision gets a bit worse, you have a point or application solution. If switching it off would break a whole set of processes that were redesigned to rely on it, you have a system solution. Independence is what makes the first two fast to deploy; interdependence is what makes the third slow, and what makes it valuable.

Why system solutions are hard, and why they matter. Point and application solutions tend to reinforce the existing system. They’re welcome because they don’t threaten anyone; they make the current way of working run a little better. System solutions, by definition, upend it. One better prediction changes what the next person should do, which changes what the person after them should do, and so on down the line. That means new workflows, new roles, new organizational structures, and resistance from everyone invested in the old arrangement.

Yet in many cases, system solutions are where the greatest returns on AI investment will be found, just as redesigned factories, not cheaper motors, were where electricity finally paid off. And because they restructure how industries operate rather than just how a single task gets done, system solutions will create winners and losers. The firms that figure out the new system first won’t just be more efficient; they’ll change the terms of competition for everyone else.

The Between Times, in other words, are not a sign that AI has failed to deliver. They’re the gap between seeing the technology and rebuilding around it, and the rest of the book is a guide to closing that gap.

[Power and Prediction] Chapter 1: Three Entrepreneurs and a Technology

~ pdawg ~ Leave a comment

Power and Prediction opens not with artificial intelligence but with electricity, and with a parable. Picture three entrepreneurs in the decades around 1900 — the stretch from the 1880s to the 1920s when factories were moving from steam power to electric power. All three have access to the same technology. What separates them is what they decide to do with it.

The first sees a point solution. Electricity is cheaper than steam and loses less energy to friction, so the obvious move is to swap the power source and leave everything else alone. The factory keeps its central shaft, its belts and pulleys, its layout. Nothing about how the plant works changes; it simply runs a little cheaper.

The second sees an application solution. Instead of one big motor driving a central shaft, why not put an individual electric drive on each machine? Now the machines are modular. If one breaks down, the others keep running. It’s a real improvement, but the factory itself is still designed as if the power came from a single source.

The third sees a system solution. Steam had dictated factory design: heavy multi-story buildings clustered around the power plant, machines arranged by their distance from the shaft rather than by the logic of the work. Electricity broke that constraint. The power source could now be anywhere, so the factory could be anything. This entrepreneur builds lightweight, single-story plants and lays out the floor around the flow of workers and materials. The productivity gains come not from cheaper power but from an entirely new way of organizing production.

Not all value propositions are created equal. The authors’ point is that the first two solutions were framed as substitution: electricity standing in for steam. That framing explains why industry was slow to adopt electricity at first; a modest saving on fuel wasn’t enough to justify ripping out working infrastructure. The lag was long. Electric motors were available in the 1880s, and by about 1919 only around half of U.S. factory power came from electricity. The productivity surge from redesigned factories didn’t arrive until the 1920s, roughly forty years after the technology first appeared. The payoff came only when some entrepreneurs recognized that electricity’s real gift was decoupling the machine from the power source, something steam made impossible or prohibitively expensive. Once that was understood, the value on offer shifted from “lower energy costs” to “a vastly more productive factory,” and system-level solutions delivered returns that dwarfed the point solutions.

The bridge to AI. The parable is the book’s organizing lens. AI, the authors argue, is at the same stage electricity was a century ago. Most current deployments are point solutions (cheaper, faster predictions dropped into existing workflows) or application solutions (new tools built around a prediction, but still plugged into systems designed for a world without them). The larger opportunity lies in asking what AI decouples. Their answer: prediction from judgment. Just as electricity let the machine move away from the power source, AI lets the prediction move away from the human decision-maker. And just as that earlier decoupling shifted the value proposition from “lower fuel costs” to “a redesigned factory,” AI’s decoupling shifts it from “lower cost of prediction” to “a vastly more productive system.”

The rest of the book is about why so few organizations have made that leap yet, and what it will take to get there.

Instant Influence from Influence by Robert B. Cialdini: Why We Fall for the Shortcut, and Why It’s Not Always Our Fault

~ stpoli ~ Leave a comment

There’s a moment near the end of Robert Cialdini’s Influence where the whole book quietly clicks into place. After hundreds of pages cataloging the specific triggers that make people say yes — reciprocity, social proof, authority, liking, scarcity, commitment — Cialdini steps back in his final chapter and asks a bigger question: why do we work this way at all?

The answer he lands on is simple. We are not built to weigh every decision from scratch. We are built to take shortcuts.

The Single Feature Stands In for the Whole

Most of the time, when we’re deciding how to respond to something, we don’t evaluate the entire situation in full detail. We grab one highly representative piece of the total picture — a single trustworthy signal — and let that one feature stand in for everything else. If something is expensive, we assume it’s good. If an expert says it, we assume it’s true. If everyone else is doing it, we assume it must be the right move.

This isn’t laziness. It’s efficiency. Our attention and mental bandwidth are limited, and if we tried to consciously analyze every input in every interaction, we’d be paralyzed by the sheer volume of decisions modern life throws at us. So we default to a rule of thumb, a trigger feature, and we react almost automatically.

The Shortcuts Are Getting More Load-Bearing

Here’s the part that feels especially relevant today. Civilization isn’t slowing down — it’s accelerating. Information moves faster, choices multiply faster, and the sheer pace of daily life keeps compressing the amount of time we have to think things through. As that pressure builds, we lean even harder on single features to make decisions for us. We count on one trusted signal to do the work that used to take careful deliberation.

There’s nothing wrong with this, in principle. If the signal is genuinely reliable — if the “expert” really is an expert, if the “five-star reviews” really reflect five-star quality, if the friend recommending something has no ulterior motive — then the shortcut is doing exactly what it’s supposed to do. It’s a reasonable trade of a little accuracy for a lot of speed, and most of the time, it works.

But Reliability Can Be Faked

The problem is that the same feature that signals trustworthiness can be counterfeited. A number can be inflated. An authority can be manufactured or borrowed. A sense of scarcity can be staged. Once someone realizes that a single trigger is enough to produce compliance, that trigger becomes a target — something worth faking rather than earning.

And this is where it gets genuinely hard. Because the shortcut wasn’t designed to filter out imitators. It was designed to save time when things are basically honest. So when someone is deliberately manufacturing the appearance of reliability — a fake review, a false urgency, a borrowed credential — it can be very difficult to tell the difference between someone giving you an honest signal and someone exploiting the fact that you’re primed to trust that signal.

Cialdini’s Closing Point

This is the note Cialdini ends on, and it’s less about self-defense and more about shared responsibility. He argues that when we recognize someone deliberately fabricating a false trigger — manufacturing fake scarcity, staging false authority, faking social proof — for their own benefit at someone else’s expense, we shouldn’t stay quiet about it. Speaking up isn’t just about protecting ourselves in that one moment. It’s about protecting the shortcuts themselves.

These automatic responses are only useful to any of us as long as the signals they run on stay honest. Every uncorrected fake erodes that a little further, making everyone slower to trust and more vulnerable to the next manipulation. Calling it out when we see it isn’t just ethical — it’s a kind of maintenance work on the mental infrastructure we all depend on.

The Takeaway

We’re never going to stop using shortcuts. The world isn’t slowing down long enough for us to abandon them, and honestly, we shouldn’t want to — they’re what make functioning at this pace possible at all. But shortcuts only work as an efficient tool for navigating an honest world. The real work isn’t eliminating our reliance on single, representative signals — it’s staying alert to when those signals have been counterfeited, and having the will to name it when they have.

Why “We” Is the Most Powerful Word in Persuasion: Unpacking Robert Cialdini’s Principle of Unity

~ stpoli ~ Leave a comment

When we think about what makes us say “yes” to someone, we usually picture classic tactics: a discount, a deadline, a bit of social proof. But there’s a subtler and, in many ways, more powerful force at work in human relationships — one that doesn’t just make us like someone, but makes us see them as an extension of ourselves. Robert Cialdini calls this the principle of Unity, and once you start noticing it, you’ll see it everywhere.

We Don’t Just Like Our Group — We Become It

The core idea behind Unity is deceptively simple: we don’t just categorize people into “us” and “them.” We actually treat “us” as part of our own identity. When someone is in our group — our family, our ethnicity, our team, our nation — we don’t just extend goodwill to them; we experience their wins and losses as partly our own.

This explains a strange psychological quirk: even a narcissist, someone who by definition struggles to genuinely care about others, will go out of his way to help a member of his in-group. Not because he’s suddenly developed empathy, but because helping “his people” feels like helping himself. The boundary between self and group blurs, and self-interest quietly does the work that altruism usually gets credit for.

Shared Identity Sells

Nowhere is this clearer than in sales and influence. Consider Ali Reda, one of the most successful car salesmen in America, who built much of his career selling within the Arab immigrant community in Michigan. His success wasn’t just about being a skilled salesman — it was that his customers saw him as one of their own. Shared ethnicity created an instant sense of “we,” and that unity did more persuasive work than any pitch could.

The same mechanism plays out in politics, sports, and even seemingly unrelated corners of life. Cialdini describes a man who had remained silent for most of his life due to a speech impairment, yet found himself shouting excitedly during a sports game — the emotional pull of being united with a crowd around a shared team was strong enough to override a lifelong pattern. Fans don’t just support a team; they merge their identity with it. That’s why a team’s loss can feel like a personal blow, and a win can feel like a personal triumph.

Blood, Family, and the Deepest Form of “We”

If shared identity is powerful, shared bloodline is the strongest version of it. Family ties create some of the most automatic and unquestioned loyalty humans experience. We help family members not purely out of altruism, but because on some level, helping them is helping ourselves — our genes, our legacy, our sense of self.

This is why one of the most effective ways to influence a stubborn person is to reframe the situation in family terms: “What would you do if this were your brother? Your daughter?” People instinctively drop their guard and their skepticism when the frame shifts from “a stranger’s problem” to “a family matter.” We simply question our family less.

Interestingly, some people expand this family-like unity to include all of humanity. This tends to happen when someone was raised by parents who were exceptionally generous and open toward others, effectively teaching them that the “family circle” can be drawn much wider than blood relations. Chiune Sugihara, the Japanese diplomat who defied his own government to issue thousands of visas to Jewish refugees during the Holocaust, is often cited as an example of this expansive sense of unity — he treated strangers with the urgency most people reserve for kin.

Even more striking is the flip side: a Nazi soldier ordered to execute prisoners in sequence deviated from his orders and shot the eleventh man instead of the tenth — because he recognized that person as a neighbor from his hometown. In a moment defined by dehumanization, a flicker of “we” was enough to override an order. It’s a chilling but powerful illustration of just how deeply unity shapes behavior, for better and for worse.

Building “We” in the Moment

Unity isn’t only about pre-existing categories like family or ethnicity — it can also be manufactured in real time. Several triggers reliably create a sense of shared identity between people who were strangers moments earlier:

  • Music and singing together synchronize people emotionally and physically, creating bonds that plain conversation rarely achieves.
  • Shared suffering is one of the strongest unifiers of all — people who go through hardship together (bootcamp, a difficult project, a crisis) often form bonds that outlast the event itself.
  • Having a common opponent naturally pulls a group closer together, whether it’s a rival sports team or a competing company.
  • Co-creation — working on something together — builds a sense of joint ownership and closeness.
  • Asking for advice, rather than feedback, is a surprisingly effective micro-technique. When you ask someone for feedback, they evaluate your work from the outside. When you ask for advice, they mentally step into your shoes and become invested in your success — and research shows people end up liking the work more after giving advice on it than after simply critiquing it.

Even our pets participate in this unconscious mirroring. Dogs and humans reciprocally yawn in response to each other — a small but telling sign of the same bonding mechanism that operates between people.

Finding Similarity, and the Danger of Blind Loyalty

One practical way to build unity with someone deliberately is to highlight genuine similarities — same hometown, same hobbies, same struggles. The more overlap two people perceive, the more the “we” identity strengthens, and the more natural liking and trust follow.

But this power cuts both ways. Once someone is firmly inside our “we,” we tend to defend them even when they’re clearly in the wrong — reflexively rationalizing bad behavior simply because the person belongs to our group. This is how unity, left unchecked, enables groupthink, favoritism, and cover-ups.

The safeguard Cialdini points to is surprisingly simple: a code of conduct, stated up front. If a group clearly establishes — before problems arise — that certain behaviors will not be tolerated regardless of loyalty, and reinforces that standard periodically, it creates a counterweight to the blind defensiveness that unity otherwise produces. The “we” feeling doesn’t have to disappear; it just needs guardrails.

The Takeaway

Unity reminds us that persuasion isn’t only about what you say — it’s about who you appear to be in relation to the other person. The more someone perceives you as part of their “we,” whether through shared identity, shared struggle, or shared creation, the more naturally trust, cooperation, and influence follow. It’s one of the oldest tricks evolution built into us, and understanding it means we can use it responsibly — and recognize when it’s being used on us.

Irvine Flowers, Done Right: A Note from the Owner of Gemma Flowers

~ pdawg ~ Leave a comment

I run Gemma Flowers, a small online florist serving Irvine and the rest of Orange County. We don’t have a storefront — never have, and that’s by design. We’re an online ordering service, full stop, and I want to write a quick honest post about why I think that’s a better way to do Irvine flowers.

The short version

Order online before 8 PM Pacific, and we’ll hand-deliver a bouquet to anywhere in Irvine that same day. The site takes orders 24/7 — after 8 PM, fresh flowers go out the next day; overnight we keep everlasting roses available.

Why online-only is the better model for flowers

A traditional storefront florist has to pay for retail rent, walk-in staff, a point-of-sale system, and a refrigerated display case sized for the showroom. Those costs come straight out of your bouquet — either the flowers are smaller, or they’re a few days older, or you’re paying for the lights more than the stems.

We skipped all of that. We run a delivery operation instead of a shop. The savings go into two things:

  • Fresher flowers. We buy direct from wholesalers two or three times a week and design arrangements the morning they go out. Most of our bouquets last 7–10 days with basic vase care.
  • No marketplace middleman. When you order from our site, your order goes to us — not auctioned out to whichever local shop will take it cheapest. The big national flower sites take 30–40% off the top before the florist on the other end even sees the order, which is why those bouquets so often look nothing like the photo.

What we deliver, and where

We cover every Irvine ZIP — 92602 through 92620, plus UCI’s 92697 — and the surrounding OC neighborhoods. Same-day inside the cutoff, next-day after (8 PM Pacific for fresh same-day).

The categories we get the most orders for:

  • Same-day delivery — birthdays, “I forgot today was our anniversary,” last-minute thank-yous.
  • Sympathy and funeral arrangements — usually under 48 hours’ notice. Standing sprays and wreaths for services, softer arrangements for the family’s home.
  • Premium and luxury pieces — anniversaries, executive gifts, special occasions. Garden roses, lisianthus, ranunculus — actual designer stems instead of carnations and filler.
  • Seasonal collections — Mother’s Day, Valentine’s, the winter holidays. These sell out, so we cap them and recommend ordering a few days ahead.

Pricing

Honest numbers:

  • $35–$45: petite arrangements — desk-sized, “thinking of you” pieces.
  • $60–$120: mid-size everyday bouquets — most of what we sell.
  • $150–$300: luxury arrangements and standing sprays for funerals.
  • $300+: larger event pieces.

Delivery fee is shown on the product page before you check out. Near the Irvine studio it’s $10; further OC cities are $20. Free pickup at CUP by Blue Hummingbird Coffee in Irvine. No marketplace “service charge,” no surprise convenience fee.

The one tip if you’re new to ordering flowers online

Order before 8 PM Pacific if you want same-day fresh flowers. Earlier is better for the fullest selection. After 8 PM, fresh bouquets go out the next morning — or grab everlasting roses anytime overnight, since we stay open for orders 24/7.

If you have a tricky delivery — hospital, gated community, a surprise to a workplace — leave a note at checkout. I read every one of them.

Bottom line

I’m biased — it’s my business. But the reason we built Gemma Flowers as an online-only operation is that it’s the model that lets us put more flower into the bouquet and less rent into the price. If you’re sending Irvine flowers to someone, that’s what you’re paying for.

If you have questions about a custom piece or a delivery I should know about ahead of time, the contact form on the site goes straight to me.

Your Data Has Opinions About How It Wants to Be Stored

~ pdawg ~ Leave a comment

Notes from Chapter 2 of Designing Data-Intensive Applications

There’s a particular kind of argument that breaks out on engineering teams roughly once a quarter: SQL or NoSQL? It’s usually framed as a question about technology, or performance, or scalability, but it almost never actually is. It’s a question about shape. Specifically: what shape is your data, and what shape do you want to query it in?

Chapter 2 of Designing Data-Intensive Applications is called “Data Models and Query Languages,” and Martin Kleppmann uses it to make a case that’s surprisingly hard to internalize: the data model you pick has more impact on how your software gets written — and how it ages — than almost any other decision. Pick the wrong shape and you’ll spend years gluing it back together with application code. Pick the right one and a lot of problems just stop existing.

Here’s the tour.

The relational model won, then NoSQL happened, then everyone calmed down

Kleppmann opens with a quick history lesson, and it’s actually load-bearing. The relational model — tables, rows, columns, joins — won decisively in the 1970s and 80s, beating out the hierarchical and network models that came before. It won because it was simpler, more flexible, and because SQL gave you a clean separation between what you wanted and how the database should get it.

Then in the late 2000s, “NoSQL” happened. Suddenly everyone was on MongoDB and the relational model was dead. Except it wasn’t — and the chapter is genuinely useful at explaining why the rebellion happened and why it cooled off.

The NoSQL pitch was real: better scalability for certain workloads, more flexible schemas, query patterns that mapped naturally to specific applications, and a way to dodge the object-relational impedance mismatch — the awkward translation layer between objects in your code and rows in your tables that ORMs spend their entire existence patching over.

But the relational model didn’t die. It absorbed. Today’s databases are increasingly hybrid: PostgreSQL has JSON columns, MongoDB has joins. The interesting question stopped being “which side are you on?” and became “which shape fits your data?”

Documents are great until your data has friends

The core argument for document databases is locality. If your data is shaped like a tree — a user with their profile, their posts, their preferences, all naturally nested inside one logical thing — then storing it as a single document means one read fetches everything. No joins. No N+1 queries. The structure of the data on disk matches the structure your code wants.

For one-to-many relationships, this is genuinely lovely. A blog post and its tags, a resume and its work history, an order and its line items — these are tree-shaped, and trees fit nicely in documents.

The problem starts when relationships go many-to-many. Suppose two users went to the same university. In a document model, you either duplicate the university name in both user documents (and now updating it is a nightmare) or you store an ID and look the university up separately (which is a join, just one your application has to do by hand). Either way, you’ve reinvented something the relational model gave you for free.

Kleppmann’s point isn’t that one model is better. It’s that the more interconnected your data is, the worse documents fit and the better relational fits. If your domain is mostly self-contained units, documents win. If everything points to everything else, you want joins.

Schema-on-read is not the same as no schema

One of the most useful clarifications in the chapter is about schemas. Document databases are often called “schemaless,” and Kleppmann pushes back on this hard. There’s always a schema — the question is just where it lives.

Relational databases use schema-on-write: the database enforces the structure when you insert data. Document databases typically use schema-on-read: the structure is implicit, enforced (or assumed) by whatever code reads the data later.

Neither is automatically better. Schema-on-read is genuinely useful when your data is heterogeneous, when the structure comes from external sources you don’t control, or when you’re moving fast and the schema is changing constantly. Schema-on-write is genuinely useful when you want guarantees, when many different applications read the same data, or when you’ve been burned one too many times by a field that was supposed to always be a string and turned out to sometimes be null, sometimes a number, and sometimes the literal string "undefined".

The flexibility of schema-on-read isn’t free. It’s a debt you pay later, in defensive code and migration scripts and 3am incidents.

Declarative beats imperative, almost always

The middle of the chapter is about query languages, and the central claim is one I think more programmers should sit with: declarative is better than imperative, and SQL is one of the great triumphs of declarative thinking.

In an imperative query, you tell the database how to find your data — loop through this, filter that, sort the other thing. In a declarative query, you describe what you want and let the database figure out how. SQL is declarative; MapReduce code (in its raw form) is imperative; CSS, surprisingly, is declarative — you describe what styled output you want, the browser figures out how to lay it out.

The advantage of declarative isn’t just elegance. It’s that the database is free to optimize. It can add an index, parallelize the query, change the join order, switch algorithms based on table sizes — all without you rewriting anything. Imperative code locks in the how, which means it can’t get faster while you sleep.

This is why ORMs that generate “good enough” SQL beat hand-tuned procedural code in most real systems: the database has more information than your application does about how to actually run the query, and it’s getting smarter every release.

When your data is mostly relationships, use a graph

The third major data model in the chapter is the graph. If documents are great for tree-shaped data and relational is great for tabular data with joins, graphs are great for data where the relationships matter as much as the things being related.

Social networks are the obvious example: who follows whom, who’s friends with whom, who reposted whom’s post. But graphs also fit naturally for road networks, recommendation systems, knowledge bases, fraud detection, and anywhere you find yourself writing recursive SQL queries with five layers of CTEs that no one will ever understand again.

Kleppmann walks through two main flavors. Property graphs (Neo4j, etc.) treat nodes and edges as first-class objects with their own properties, queried with languages like Cypher. Triple-stores (RDF, SPARQL) model everything as (subject, predicate, object) triples, which is conceptually clean and has roots in the Semantic Web movement. Both can express things that would be miserable in SQL — like “find all the people connected to me by at most three hops who work at companies headquartered in cities I’ve visited” — in a few lines.

The historical note he ends on is great: most of these graph query languages descend from Datalog, a declarative logic programming language from the 1980s. Half of computer science is rediscovering ideas from the 1980s with better marketing.

The takeaway

Three data models — relational, document, graph — and they’re not really competitors. They’re tools for different shapes.

  • Tree-shaped, mostly self-contained data with one-to-many relationships → documents.
  • Tabular data with lots of many-to-many relationships → relational.
  • Highly interconnected data where the relationships are the point → graph.

Most real systems eventually use more than one. Your user accounts live in Postgres, your activity feed lives in a document store, your recommendation engine reads from a graph. That’s not a failure of architecture; it’s the architecture working as intended.

The deeper lesson is the one Kleppmann keeps coming back to: data outlives code. The schema you pick today will still be shaping decisions five years from now, long after the framework you used to build the API has been replaced. Choosing the right model early is one of the cheapest performance and maintainability wins in software.

Choose for your data’s shape, not for the conference talk you saw last week.

The Three Words Every Backend Engineer Should Tattoo on Their Forearm

~ pdawg ~ Leave a comment

Notes from Chapter 1 of Designing Data-Intensive Applications

If you’ve spent any time around backend engineers, you’ve probably noticed they love to argue. Postgres or MongoDB? Kafka or RabbitMQ? Microservices or “the modulith”? Most of these debates feel like they’re about technology, but they’re almost never really about technology. They’re about tradeoffs — and the tradeoffs only make sense when you know what you’re optimizing for.

Martin Kleppmann opens Designing Data-Intensive Applications by giving us the vocabulary to have those arguments properly. Chapter 1 is called “Reliable, Scalable, and Maintainable Applications,” and those three adjectives are the entire point of the book. Get them right, and the rest of the 500 pages is essentially a tour of how different systems make different bets in service of those three goals.

Here’s what the chapter actually says, and why it’s worth slowing down on before racing into the chapters about replication and consensus.

What even is a “data-intensive” application?

Kleppmann’s framing in the first few pages is small but important: most of the systems we build today are not bottlenecked by raw CPU. They’re bottlenecked by the amount of data, the complexity of it, or the speed at which it changes. A web app that serves a million users isn’t doing hard math. It’s juggling state — reading it, writing it, caching it, indexing it, replicating it, keeping it consistent enough to be useful and inconsistent enough to be fast.

These applications are built out of remarkably standard parts: databases, caches, search indexes, message queues, stream processors, batch processors. The interesting engineering question isn’t usually “which database?” It’s “how do these pieces fit together for this workload?” Two apps with identical tech stacks can have wildly different architectures because they’ve answered that question differently.

That sets up the rest of the chapter. If your job is gluing data systems together, what are you actually trying to achieve?

Reliability: keep working when things go wrong

The first goal is reliability — and Kleppmann gives a definition that sounds obvious but is genuinely useful: a reliable system continues to work correctly even when things go wrong.

The key distinction here is between a fault (one component misbehaving) and a failure (the whole system stopping). The job of a reliable system isn’t to prevent faults — that’s impossible — it’s to prevent faults from cascading into failures. That’s what “fault-tolerant” means.

Faults come in three flavors:

  • Hardware faults. Disks die, power cuts out, networks flake. We’ve been dealing with these for decades, mostly through redundancy: RAID arrays, dual power supplies, multiple availability zones. This is the easy category, in the sense that the failure modes are well understood.
  • Software errors. Bugs, runaway processes, cascading failures where one slow service takes down everything that depends on it. These are nastier because they can hit every replica simultaneously — your fancy redundancy won’t save you if all three nodes have the same bug.
  • Human errors. And here’s the punchline: humans cause more outages than hardware. The defenses are good abstractions, sandboxed environments for testing, telemetry that catches problems early, and — crucially — making it easy to roll back when someone inevitably ships something broken at 4pm on a Friday.

It’s tempting to skip reliability work on “non-critical” applications, but Kleppmann pushes back on that: the cost of losing user trust usually exceeds the cost of building things properly the first time. A photo app isn’t life-or-death, but if it loses your wedding photos once, you’re never opening it again.

Scalability: cope with growth

Scalability is the one everyone thinks they understand and almost no one defines properly. Kleppmann’s framing is that “scalable” isn’t a property a system has or doesn’t have — it’s a question, and the question only makes sense if you specify two things: what you mean by load, and what you mean by performance.

Load is whatever parameter actually pressures your system. For a web server it might be requests per second. For a cache it might be the hit rate. For a database it might be the read/write ratio. The chapter’s famous Twitter example shows why this matters: serving home timelines is a hard problem, but the right solution depends entirely on whether you optimize for the read path (fan-out on write, materialize each user’s timeline) or the write path (fan-in on read, query everyone’s posts when the user opens the app). Twitter actually switched approaches as their workload changed. Same problem, different load characteristics, different architecture.

Performance is the other side. And here Kleppmann lays down what I’d argue is the single most important graph in backend engineering: don’t use averages, use percentiles. A system with a 100ms average response time can still be miserable to use if the slowest 1% of requests take 10 seconds. Tail latencies — p95, p99, p999 — are what users actually feel, and they tend to disproportionately hit your most engaged customers, the ones who make the most requests and therefore have the most chances to roll the bad-luck dice.

Once you know your load and your performance target, scaling becomes a design problem. You can scale up (bigger machine) or out (more machines), and the right answer depends on the workload. There is no universal scalable architecture. Anyone selling you one is selling you something.

Maintainability: make it livable

The last goal is the one engineers love least and pay for most. Most of a system’s lifetime cost is not in writing it; it’s in keeping it running, evolving it, and onboarding new people to it. Maintainability is the property that lets future-you (or the person who replaces you) keep the lights on without losing their mind.

Kleppmann breaks it into three sub-principles:

  • Operability. Make it easy for the operations team to keep the system healthy. Good monitoring. Predictable behavior under load. Useful logs. Documentation that exists. Default behaviors that make sense. The classic “it works on my machine” failure is an operability failure.
  • Simplicity. Manage complexity. Every system accumulates accidental complexity over time, and the main weapon against it is good abstractions — taking something messy and giving it a clean interface. The opposite, which Kleppmann calls a “big ball of mud,” is what happens when you skip this and let everything tangle into everything else.
  • Evolvability. Make it easy to change. Requirements always change. The org changes, the product changes, the regulations change, the scale changes. Systems that can’t evolve get rewritten, and rewrites are expensive and dangerous.

If you’ve ever worked on a codebase where every change feels like defusing a bomb, you’ve experienced the absence of all three at once.

The takeaway

Reliability, scalability, maintainability. Three words, and the entire book is essentially “how do specific tools and techniques affect these three properties for specific workloads?”

What I find genuinely useful about this chapter is that it gives you a way to evaluate technical decisions without falling into religious wars. When someone says “we should use Kafka,” the question isn’t whether Kafka is good. It’s: which of these three properties does Kafka improve, by how much, for our workload, and at what cost to the others? Sometimes the answer is “a lot, cheaply, do it now.” Sometimes the answer is “not really, and it’ll add a fourth on-call rotation.”

There are no silver bullets. There are only tradeoffs against a specific set of requirements — and the first job of any backend engineer is being able to say what those requirements are.

That’s the real lesson of Chapter 1. The rest of the book is just receipts.

Commitment and Consistency: Why We Stick to Our Decisions (Even When We Shouldn’t)

~ stpoli ~ Leave a comment

A summary of Chapter 7 Commitment and Consistency by Robert Cialdini

In Chapter 7 of Influence, Robert Cialdini explores one of the most powerful psychological principles shaping human behavior: commitment and consistency. At its core, this principle explains that once people make a choice or take a stand, they feel internal and external pressure to behave consistently with that commitment. While this tendency can help us stay disciplined and reliable, it also makes us highly susceptible to manipulation.

One of the most fascinating aspects of this principle is that the act of choosing matters more than the outcome itself. When people feel they’ve freely made a decision, they begin to align their identity with that choice—and will go to surprising lengths to justify and maintain it.

Commitment Strengthens Identity

A powerful modern example of this principle comes from Amazon, which offers employees up to $5,000 to quit their jobs. At first glance, this seems counterintuitive—why would a company pay people to leave? The goal isn’t actually to encourage resignations, but to strengthen commitment among those who stay. By giving employees a clear option to leave—and attaching a monetary incentive to it—those who remain feel more confident and committed in their decision. They didn’t stay by default; they actively chose to stay.

This highlights a key idea: commitment is strongest when it feels voluntary.

The Power of Saying “Yes”

Cialdini also illustrates how even small verbal commitments can dramatically influence behavior. In one experiment, when a person’s belongings were stolen in public, bystanders rarely intervened. However, when the owner simply asked someone nearby to “watch their things,” the results changed dramatically—those same bystanders almost always chased down the thief.

Why? Because they had made a commitment, even a small one. Once they said “yes,” they felt compelled to act in alignment with that statement.

Consistency as a Mental Shortcut

One reason people rely so heavily on consistency is that it simplifies life. Constantly reevaluating decisions requires mental effort, so once we’ve made a choice, we prefer to stick with it rather than reconsider.

Cialdini shares a striking example: he and a psychologist friend attended a presentation promoting meditation as a solution to personal problems. His friend effectively dismantled the presenters’ arguments in front of the audience, exposing weak logic. Despite this, many attendees still signed up and paid a deposit immediately afterward.

When asked why, one participant admitted they understood the criticism—but chose to commit right away so they wouldn’t go home and rethink the decision. They wanted the solution to work, so they locked themselves into it.

This shows how commitment can override logic. People don’t always want the truth—they want consistency with what they’ve already started to believe.

How Businesses Use Commitment

Toy manufacturers have long exploited this principle. They advertise a must-have toy before Christmas, creating excitement and prompting parents to promise it to their children. But when the holiday arrives, the toy is “sold out.” Parents, feeling committed to their promise, buy alternative gifts of equal value. Later, when the toy becomes available again, many parents purchase it anyway—effectively buying twice.

Salespeople use similar tactics. For example, a car salesperson might ask, “If the price is right, would you buy this car today?” Once the customer agrees, they’ve made a verbal commitment. This makes it much harder to back out later, even if the deal changes.

Charities also take advantage of this tendency. A common tactic is asking, “Are you having a good day?” If someone says yes, they’re more likely to donate—because refusing would feel inconsistent with their stated positivity and good fortune.

Small Steps Lead to Big Commitments

The principle of commitment is especially powerful when it starts small. During the Korean War, Chinese captors used this strategy on American prisoners. Instead of forcing large ideological statements, they began with small, seemingly harmless agreements—like acknowledging that no country is perfect. Gradually, prisoners were led to write essays critical of the United States, and eventually, some became cooperative with their captors.

This gradual escalation shows how small commitments reshape self-image over time.

Similarly, in a study, homeowners who agreed to place a small “Drive Carefully” sign in their yard were later far more likely to support unrelated causes, like signing a petition for environmental protection. Their initial action changed how they saw themselves—as civic-minded individuals—which influenced future behavior.

Writing and Effort Deepen Commitment

Cialdini emphasizes that written commitments are especially powerful. Even copying a statement—without initially agreeing to it—can influence beliefs over time. When these statements are made public, the effect is even stronger, as people feel pressure to maintain a consistent image in front of others.

Effort also increases commitment. This explains why people who go through intense initiation processes, such as fraternities, often become more loyal. The more someone sacrifices to achieve something, the more they justify its value.

Commitment Without Pressure

Interestingly, commitment works best when people feel they’ve made the choice freely. External pressure or threats may produce short-term compliance, but they don’t create lasting change. When people believe they’ve chosen something on their own, they internalize it—and continue the behavior even without outside influence.

A study on energy conservation demonstrates this. Residents were told their names would be published in the newspaper as environmentally responsible citizens if they reduced energy use. This worked—but even after being told their names would not be published, they continued conserving energy at even higher levels. Why? Because their self-image had already shifted. They now saw themselves as energy-conscious individuals.

The “Foot-in-the-Door” Effect

Another powerful tactic is getting someone to agree to something small, then increasing the demand later. For example, researchers asked participants if they’d be willing to volunteer for a study. After they agreed, they were told the study would take place at 7 a.m. Surprisingly, many still followed through. Their initial commitment made them more likely to accept less appealing conditions.

Avoiding Foolish Consistency

While consistency can be beneficial, Cialdini warns against becoming “foolishly consistent.” People often stick with decisions that no longer serve them simply to avoid feeling inconsistent.

To protect ourselves, we should:

  • Pay attention to our gut feelings when something doesn’t feel right
  • Ask ourselves: Would I make this same decision if I had all the information upfront?
  • Recognize when someone is using small commitments to lead us into bigger ones

Consistency should serve us—not trap us.

Final Thoughts

The principle of commitment and consistency is powerful because it operates internally. Once activated, it doesn’t require external pressure—people will find their own reasons to stay aligned with their past decisions.

Understanding this principle helps us in two ways:

  1. We can become more aware of how others influence us
  2. We can use commitment intentionally to build positive habits and identities

At its best, consistency creates integrity and discipline. At its worst, it can lead us to defend poor decisions and ignore better options. The key is knowing when to stay committed—and when to change course.

The Power of Less: How Scarcity Shapes Every Decision You Make

~ stpoli ~ Leave a comment

A Summary of Chapter 6 from Influence by Robert B. Cialdini

What if the secret to wanting something more had nothing to do with what it actually was and everything to do with how available it seemed? In Chapter 6 of Influence, Robert Cialdini unpacks one of the most quietly devastating forces in human psychology: scarcity. The principle is simple. We place greater value on things that are rare, fleeting, or at risk of being taken away. And the less available something becomes, the more desperately we want it.

The Art of the Almost-Lost Deal

Consider a divorce lawyer who spent years struggling to get couples to agree on settlement terms. Despite presenting identical proposals, she found clients stubbornly resistant — until she made one subtle change in how she framed the moment of decision. The old version went: “All you have to do is agree to the proposal, and we will have a deal.” The new version flipped the sequence: “We have a deal. All you have to do is agree to the proposal.”

The result? A near-perfect success rate. The reason is rooted in loss aversion. In the original phrasing, clients imagined themselves agreeing and therefore potentially giving something up. In the revised phrasing, the deal already existed in their minds — and refusing meant losing it. People will fight far harder to keep something they believe they already have than to gain something new. The lawyer didn’t change the terms. She changed what was at stake.

Midnight Lineups and Louis Vuitton Purses

Apple understands scarcity better than almost any company on earth. When a new iPhone launches with “limited supply” in stores, it does not simply create demand — it manufactures urgency. Long lines form overnight. Social media fills with stories of people who camped out, traded favors, and made bizarre sacrifices just to be among the first to get their hands on the device.

One story stands out particularly well. A woman waiting in line spotted someone just two spots ahead of her and offered to trade her Louis Vuitton handbag for their place in line. The rational mind would question this trade. But in a scarcity mindset, logic yields to the terror of missing out. The possibility of not getting the iPhone — of losing the opportunity — outweighed the objective value of a luxury bag. That is the power Cialdini is describing: not just desire, but the fear of deprivation.

Loss Looms Larger Than Gain

Research confirms what common experience hints at: the pain of losing something is significantly more motivating than the pleasure of gaining something of equal value. In one striking study, team members were found to be 82% more willing to cheat in order to prevent their team from losing status than they were to cheat in order to gain it. The asymmetry is striking. We are not rational optimizers seeking the best outcome — we are loss-averse creatures wired to protect what we already have.

This is why companies that frame their messaging around what customers stand to lose — rather than what they might gain — consistently outperform those that don’t. Health organizations encouraging cancer screenings have found dramatically better results when they ask people not to lose the chance to be healthy, to retain the ability to be present for life’s special moments, rather than simply promoting the benefits of early detection. The framing of loss is simply more compelling to the human mind.

The eBay Dad, the Countdown Clock, and the Three-Call Con

Scarcity operates through two distinct triggers: limited quantity and limited time. A father selling his collection of rare trading cards on eBay discovered this firsthand. When he listed all his cards at once, bids remained modest and interest was lukewarm. But when he staggered the listings — releasing one card at a time with gaps between each — the sense of rarity transformed his results entirely. The same cards, the same buyers, but a completely different outcome driven by perceived scarcity.

Deadlines exploit the same mechanism. When a window of opportunity appears to be closing, people stop deliberating and start acting. This urgency, Cialdini warns, is precisely what unscrupulous salespeople exploit. One chilling example involves a fraudulent investment scheme built on a “three-call method.” The first call is purely informational, delivered under the name of an impressive-sounding company. The second call reports remarkable profits — but regretfully notes that the investment window has closed. Then comes the third call: an exclusive opportunity, available only now, for a limited time. One man, caught in this manufactured urgency, handed over his entire life savings. The genius of the scheme was not greed — it was the engineered fear of missing out.

Freedom, Toddlers, and the Psychology of Reactance

Why does scarcity work at all? Cialdini points to two deeply rooted psychological forces. The first is a reasonable heuristic: things that are hard to obtain are often genuinely better. Rare materials, exclusive access, and limited editions frequently do represent superior quality. The second force is more primal — we hate losing our freedom to choose.

This psychological reactance — the instinct to push back when options are restricted — explains two of life’s most famously difficult developmental stages. At around age two, children first discover that they have independent will. Take something away, and they want it fiercely. Teenagers experience a second surge of this same impulse as they form their identities against the limits imposed by parents and society. Both stages are marked not by irrationality, but by an acute sensitivity to the loss of autonomy.

New Scarcity Hits Hardest

Cialdini closes with a crucial nuance: it is not just scarcity that inflames desire, but newly emerging scarcity. When something that was once plentiful starts to disappear, people react far more intensely than if it had always been rare. The sense of loss is compounded by the contrast with what was previously available. This is why rising restrictions, shrinking stock, and expiring offers trigger such powerful responses — the mind is not just registering scarcity, it is registering loss in motion.

Understanding scarcity means recognizing it everywhere — in the countdown timer on a checkout page, in the “only 3 left in stock” label, in the exclusive offer expiring at midnight. These are not coincidences. They are carefully engineered triggers aimed at the most ancient part of our decision-making brain: the part that is far more afraid of losing than it is excited about winning.

The Hidden Power of Authority: Chapter 5 from Influence by Robert B. Cialdini

~ stpoli ~ Leave a comment

From the moment we are born, we are conditioned to listen to authority. Parents, teachers, doctors, governments — our entire social fabric is built on a hierarchy of trust. In Chapter 5 of his book Influence: The Psychology of Persuasion, Robert B. Cialdini explores how deeply this instinct is wired into us, and how easily it can be exploited. The findings are eye-opening, occasionally disturbing, and profoundly relevant in both our personal and professional lives.

Authority Outperforms Even the Best Incentives

Cialdini presents a striking example that illustrates just how powerful perceived authority can be. In a donation study, researchers compared two approaches: giving potential donors a small sweet as a goodwill gesture before making the ask, which is a well-established compliance technique, versus simply having the CEO of the organization make the request. The result? The CEO’s direct appeal generated more donations than the sweetened offer. This finding alone shows how authority reigns over even the small material gestures for compliance. When an authority figure steps forward, their presence alone carries more weight than tangible rewards.

The Milgram Experiment: Obedience Pushed to Its Limits

Perhaps the most chilling evidence Cialdini draws upon is Stanley Milgram’s now-infamous obedience study. In this experiment, participants were instructed by a researcher in a white lab coat to administer electric shocks to another person whenever they answered a question incorrectly. The shocks were not real, but the participants did not know that. As the voltage levels escalated, the person on the receiving end — an actor — could be heard pleading and crying out in pain, eventually going completely silent. Yet the majority of participants continued to administer shocks simply because an authority figure told them to. Even when their conscience screamed at them to stop, the presence and insistence of someone in a position of authority overrode their better judgment. This experiment reveals something deeply unsettling: our deference to authority is not just a social nicety — it can override our own moral instincts.

Authority Is Learned — And That Makes It Universal

Why are we so susceptible? Cialdini argues it is because compliance with authority is deeply conditioned from childhood. We are taught from our earliest years that listening to parents, teachers, and elders keeps us safe and leads to positive outcomes. This conditioning then extends seamlessly into adult life — we defer to bosses, doctors, legal systems, and government bodies. The behavior is so ingrained that it becomes automatic. We do not stop to critically evaluate each instruction from a perceived authority; we simply comply. What made us good, safe children ultimately can make us vulnerable adults in the wrong hands.

Symbols of Authority: Titles, Clothing, and Status Trappings

One of the most fascinating aspects of this chapter is Cialdini’s revelation that we do not even need real authority — the symbols of authority are enough. Titles, uniforms, and status markers like an expensive car or a large home trigger the same automatic compliance response as genuine expertise or position. In one telling example, a television commercial featured an actor who clearly stated at the beginning that he was not a real doctor — yet proceeded to behave exactly like one throughout the ad. Despite this upfront disclaimer, sales for the company increased significantly after the commercial aired. Viewers’ minds defaulted to the visual and behavioral cues of a doctor, and the logical disclaimer failed to override the emotional response.

Hackers and the Art of Social Engineering

Authority does not only operate in formal, top-down settings. Cialdini highlights how skilled manipulators deliberately exploit authority cues to bypass even security systems. A notable case involved a successful hacker who gained access to a bank’s secure areas not through any technical breach, but by having accomplices pose as janitors and maintenance workers. Bank employees — conditioned to grant access to people who appeared to belong there and who carried an implicit sense of routine authority — allowed them into restricted areas containing sensitive private information. The disguise was not a military uniform or a lab coat; it was simply the quiet, unassuming authority of someone who “looked like they were supposed to be there.”

A Note for Leaders: Knowledge Over Command

Cialdini closes this exploration with an important insight for managers and leaders. While people will comply with authoritative demands, they do not enjoy being bossed around. Compliance driven by positional power alone breeds resentment and disengagement. What truly resonates with people is authority rooted in knowledge and expertise. When a leader demonstrates genuine understanding, shares insight, and earns trust through competence, people do not merely comply — they listen willingly and act with greater conviction. The most effective leaders understand this distinction: authority commands, but expertise inspires.

Trustworthiness: The Authority Multiplier

Beyond titles and credentials, Cialdini identifies trustworthiness as one of the most powerful amplifiers of authority. And the most counterintuitive way to build trust? Admitting a mistake. When someone in a position of authority voluntarily acknowledges a flaw or misstep before presenting their case, it signals honesty — and that signal disarms skepticism in a way that a perfectly polished pitch never could.

Warren Buffett masterfully uses this approach in his annual letters to Berkshire Hathaway shareholders. Before walking investors through the year’s gains and accomplishments, Buffett openly acknowledges mistakes he made — poor investments, misjudgments, errors in strategy. This candid admission does not undermine his authority; it supercharges it. By the time he pivots to Berkshire’s strengths and highlights of the year, readers are fully convinced they are hearing from someone honest. The vulnerability earns him credibility that no amount of bragging ever could.

This same principle plays out in consumer behavior. Research shows that products with a perfect five-star rating are actually less persuasive than those with a slightly imperfect score. When every review is glowing, people grow suspicious — it feels manufactured. But when a product has a few honest criticisms alongside strong praise, buyers feel they are getting a realistic picture, and they trust the positive reviews more as a result. Even more compelling is how reviewer credibility works: when a reviewer admits their own mistake or limitation, readers are far more likely to trust and act on that person’s recommendation. The small concession makes everything else they say feel more genuine.

Protecting Yourself: How to Avoid Being Wrongly Influenced

Understanding authority’s influence is not just an academic exercise — it is a practical tool for protecting yourself from manipulation. Cialdini offers two key reminders for navigating a world filled with authority cues, both real and manufactured.

First, always pause and ask: Does this person actually have credentials in the field they are speaking about? A confident tone, an impressive title, or a professional appearance can trigger automatic deference — but none of those things confirm genuine expertise. A doctor speaking about financial investments, or a celebrity endorsing a medical product, may carry all the trappings of authority with none of the relevant knowledge. The habit of asking “what qualifies this person to speak on this specific topic?” is a simple but powerful filter.

Second, be aware that the “small mistake” tactic is itself a tool of influence. Just as Warren Buffett uses it authentically to build genuine trust, savvy marketers, salespeople, and manipulators have learned to deploy it strategically. A calculated admission of a minor flaw — one that costs them nothing — can be used to make everything else they say feel more credible. When someone leads with a small concession and then immediately pivots to a strong, persuasive case, it is worth asking whether the vulnerability was genuine or engineered. Trustworthiness is one of the most powerful forces in persuasion precisely because it feels so unscripted. Knowing that it can be scripted is your best defense.

Final Thoughts

Chapter 5 of Influence is a masterclass in understanding one of the most powerful forces shaping human behavior. Authority — whether real or merely perceived — has the capacity to make ordinary people donate more generously, ignore their own moral compass, believe actors over their own reasoning, and open doors they should keep closed. And when combined with the perception of trustworthiness, its pull becomes nearly irresistible.