I lead the AI Infrastructure and Applied Engineering Developer Relations teams at Google Cloud. AI Infrastructure is GKE and the hardware beneath it—the accelerators, orchestration, and drivers that keep large training, fine-tuning, and inference jobs running—and the work is helping teams get the most out of the world's most capable AI hardware. Applied Engineering is a Developer Relations program I started in 2023: Google Cloud engineers co-engineer a mission-critical solution alongside a customer's own team, on the customer's project. It is not consulting and it is not sales. Self-sufficiency is the deliverable, and the engagement worked only if the customer no longer needs us when it ends. Across both teams, agents are arriving in production at real scale, and my teams build the governance, security, and performance foundations that let enterprises run them with confidence. I have spent more than 25 years building internet infrastructure, application platforms, and information security systems, and the work has always come back to the same thing: technology only matters as much as the people who run it. I started in open source with Perl, and some of that code still ships with macOS today. At Pivotal I championed Cloud Foundry to enterprise engineering teams around the world; at WhiteHat Security I rebuilt the software development lifecycle and moved the company to continuous delivery; at Socialtext I led engineering on one of the first commercial wikis. The lesson that held across all of it: the most resilient systems come from teams that are empowered, productive, and trusted to do the work. Today I am focused on [**The Agentic Manifesto**](https://agenticmanifesto.ai), an operating model for a world of non-deterministic, autonomous AI. Building for that world means polyglot tool ecosystems on standardized protocols, and development practices that plan for emergent behavior, continuous tuning, and automated governance rather than fighting them. I speak and write about modern architecture, DevOps, and the cultural shifts that new technology demands of the teams adopting it. You can find me at conferences like KubeCon, OSCON, and QCon, or right here on this blog. Beyond my professional life, I live in Pittsburgh with my wife and family. For a detailed look at my professional background, my [resume is available here](/resume). --- Don't promote your most talented technologists into people management. Promote them into technical leadership roles instead. You'll keep happy, engaged leaders and avoid the perils of the [Peter Principle](http://en.wikipedia.org/wiki/Peter_Principle). It's a common policy in hierarchical organizations to promote top performers into people management. Your best programmer comes up with great ideas, the team listens to her, she can fix anything. It's review time and what happens? She's offered a job in management. What are her new responsibilities? Paperwork, reviews, operational alignment summits, budgets. How much code is she writing now? Not much and she's probably miserable. > "I don't want to stop coding." — Almost every good programmer, ever. ![A giant cup of I'm the fucking boss.](/images/posts/a-call-for-more-tech-leadership/boss.jpg) _This was a Christmas present from a member of my team in 2014._ There's another alternative. Technical leadership. Here are some common tech leadership roles:
Lead Engineer
You're coding everyday and helping others to code better, too. You have deep experience with the tools and technologies, as well as the problem domain, and you use that to inform internal system design. You keep the code healthy.
Software Architect
You're designing the story arc of how this software fits into its problem domain. You have visibility into the long-term plans for the software, and strong technical skills to work with engineering to build something that keeps the software focused. You keep the software healthy.
CTO
The most senior architect. You know what every product line is aiming for. You know how it fits into the long-term story of the product and the industry. You share this insight with your architects and leads to give them vital context for making great choices. You keep the company healthy.
## This is the intersection of technology and people. > People are at the heart of tech leadership roles. It's a socio-technical role, a mix of people and technology. [Hacking on one and not the other will cause the performance of both to suffer.](/your-software-is-made-of-people) This is the critical glue missing from most engineering organizations where engineers are promoted to managers. This is where technical leadership plays its key role. **You're not managing people: you're leading them.** Why is there a gap? Because newly-promoted managers now have too many jobs. Promoting a tech lead into a manager forces them into the problem of having too many roles. Their boss is expecting them to be a full-time people manager, but their team depends on their technical leadership. They'll either have to choose one or do both poorly, for lack of time, which will eventually force their hand into choosing people management. Again, because _their_ boss is expecting them to do that role. Now there's a void in technical leadership. In a healthy organization the tech lead (lets say a Principal Architect) and the people manager are a powerful pair. When they're working together the people on their team are taken care of at every level. The team is comfortable because they know what to build and why, and that they're building on the right path. They're also getting the individual attention of their manager. ## Tech leadership is a [force multiplier](https://en.wikipedia.org/wiki/Force_multiplication). As a technical leader you aren't always writing code. If you're a recognized leader chances are you already know that. You spend a lot of time helping others. Teaching, mentoring, reviewing code, standing at the whiteboard and explaining the permissions model for the hundredth time. Or whatever your pet thing is… You may not always be pressing many keys and causing an implementation to occur, but for the people who are you're a valuable asset. You help them do it better. You help them avoid the pitfalls and mistakes you've already made five times in your career. This work is vital to the health of an organization. It has a direct impact on productivity and happiness. You are helping each team member avoid pain, and you're helping the organization avoid pain. How is this a force multiplier? The evidence is proven over time. Applying the skill of technical leadership is a force multiplier in several ways. **If any of the following apply to you, you might be a tech lead:** - Your experience helps engineers at all skill levels create solutions that last longer, are easier to maintain, and respond to change better. - You've thought through integration problems so there are fewer surprises. - You understand the long-term goals for the software so your short-term solutions don't paint everyone into an architectural corner. - Your team is constantly learning from you so retention rates are high because everyone is growing. - When a customer is upset you can code up a solution which keeps the team focused on their planned work. - You've been planning ahead for the next big development effort — have five prototypes ready for comment by the time the team is ready — so nobody feels lost in the face of a new challenge. This is how a technical leader is a force multiplier: enhancing the attributes of your organization to make it more effective than others. It's like skill boosts in role-playing games (be honest, if you're reading this you know exactly what I mean). ## Lead by example. Technical leadership is the role best suited for my skill set. I was fortunate enough to have been supported by a fantastic people manager at my current company. With support I used everything in my technical leadership toolbox to build teams of outstanding engineers, create a culture for them to thrive in, write the story of our next great architectural revolution, and lead the team down that path. I love this work. I'm passionate about it. As a programmer who always wants to code, I don't get to do it as much as I'd like, but when when I do it's usually pairing with someone on my team. As a leader I get to work directly on the problems getting in the way of my team, very real people-problems, and resolve them. I do that with the support of people management. One of the core values in my team's culture is transparency. I'll exercise that by sharing a piece of my self evaluation from last fall. It's the part where I'm supposed to say what I ought to work on for the next year. This is a high-level view of my approach to technical leadership: ### What should the focus be for the next review period? I'd will continue to focus on the areas I feel are the greatest value to WhiteHat. This has been my focus for the last year and will remain so. Specifically: **Empower Others** — I will continue to create space and opportunity for the members of my team, my organization, and anyone in the company to contribute at their best level. **Challenge Assumptions** — I will continue to challenge the things we do, how we do them, and why we do them in a relentless effort to eliminate waste in our organization. **Eliminate Pain Points** — I will continue to learn about others' work in order to find resolution for things that are difficult and painful in their daily work. I will empower my team to find ways to automate most of those pain points away, or challenge assumptions in our process and workflow which may have created them. **Influence Culture** — I will continue to be the change I want to see at WhiteHat, and be myself with no bad politics or hidden agendas, in order to help create the culture of trust, learning, productivity, fun, and reward I want to work in; that I want to recruit my friends to work in. **Lead Technology and Architecture** — I will continue to be a leader, mentor, and teacher to my team members and organization members. I will take an active role in leading the development and design of our software and operational architectures to ensure we make aggressive strides toward modern, sustainable solutions that meet our customers' needs. By this time next year I will have been successful if the Engineering organization is delivering high quality software that delights our customers. I expect my team to have delivered an infrastructure and web application framework that puts us in the position to aggressively compete in our market, to have delivered several key features which empower WhiteHat to make impressive advancements within our industry and in comparison to our key competitors. ## We need a tech leadership track. I'm sure you've heard this before: _"we're going to put you on the management track."_ The management track is a career advancement concept designed to mold high performers into managers. Within engineering we need a new, parallel track. We need a technical leadership track. Top performers who want to remain technical and advance their career should have the option. Organizations are in desperate need of highly qualified, deeply experienced leaders making strategic decisions for the health of their software. A technical leadership track is the best way to source that talent from within. **If you are a mid to senior level engineer** right now, and you don't want to advance to management, I want to encourage you to advance to technical leadership instead. Insist on a promotion to Lead Engineer, then Architect, then Principal Architect. **If you run an engineering department** and want to offer career advancement that keeps your top performers engaged and happy, don't force them into management. Create an advancement track designed to build a strong technical leadership pipeline. Your software will be better for it. Find your best technologist and promote them to Director of Architecture. Your new Director should have people managers in her peer group. This creates space for the tech leaders in your organization to grow. Build your technical leadership team under her in the hierarchy, but let them keep working with the teams their leading in day-to-day work. Let them continue to do what they're great at. **This is how you make technical leadership just as important as people management.** ![It's good to be back in the shire.](/images/posts/a-call-for-more-tech-leadership/feet-up-boss.jpg) _I **am not** a manager but I love this mug._ --- Twenty-one years ago I was writing wiki engines. In 2005 I worked at [Socialtext](https://en.wikipedia.org/wiki/Socialtext), one of the first commercial wikis, on the then-radical idea that a team could keep its shared knowledge on a page anyone could edit. That same year, for a running contest on Ward Cunningham's original wiki, I got a working wiki down to four lines. So when Andrej Karpathy posted his [LLM Wiki idea](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) in April, and Google shipped a spec that formalized a slice of it, I didn't read it as news. I read it as a thread I'd been pulling on for two decades, with the same knot still tied in it. ## The Wiki I Fit in an Email Signature The [Shortest Wiki Contest](https://wiki.c2.com/?ShortestWikiContest) asked for the smallest source code that still ran a real wiki: automatic linking, editable pages, the whole contract. People got it to a few dozen lines. Building on the work done on FleaWi, PeeWee, and mostly PeWi, I got it to four, and called it [SigWik](https://wiki.c2.com/?SigWik) because it fit in an email signature, 80 columns wide. Nick Clark named it, for the four lines. Doug Merritt's reaction on the page is still the best code review I've gotten: _Ok, now I am amazed._ Here it is, 218 characters of Perl and shell, reproduced from the contest page: ```perl #!/usr/bin/perl use CGI':all';path_info=~/\w+/;$_=`grep -l $& *`.h1($&).escapeHTML$t=param(t) ||`dd<$&`;open F,">$&";print F$t;s/htt\S+|([A-Z]\w+){2,}/a{href,$&},$&/eg; print header,pre"$_
",submit,textarea t,$t,9,70 ``` I'm not showing this to brag about a stunt (or maybe I am). I'm showing it because it makes the boundary concrete. SigWik is the _format_: a folder of files, one file per page, links found by pattern. That part is genuinely small, a clever afternoon. Ok, I'll be honest, several weeks of iterating. What four lines can't contain is the part where a human comes back next week, sees the page is wrong now, and fixes it. The format is the easy half. It always was. ## What Karpathy Proposed, and What Google Kept Karpathy's idea was never "a folder of markdown." Read the gist and it's the operations that matter: the model _incrementally builds and maintains_ a persistent, interlinked wiki between you and your raw sources. His framing sticks with me: _Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase._ Knowledge gets compiled once and kept current, instead of re-derived from scratch on every query the way retrieval-augmented generation does it. But the pitch has a part that isn't a folder at all. It describes ongoing upkeep, not just a layout. Karpathy spends a section on it: periodically, ask the model to health-check the wiki, hunting contradictions, stale claims a newer source overtook, and orphans nobody links to. He calls it the lint step. It's the maintenance loop, and it's what keeps the wiki from becoming a museum of things that used to be true. Now Google's [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md). OKF is a real, minimal, well-designed spec, version 0.2 as I write this, and standardizing the container is worth doing: a bundle is a directory, one file is one concept, the file path is the ID, and links are a plain graph. Two reserved files, `index.md` and `log.md`, do what they did in the gist. The container Karpathy sketched, Google wrote down. The spec kept the structure. It left out the recurring check. ## A Field Is Not a Process OKF gives you a `modified` field. It records a fact: here's when this concept was last written. Nothing notices that a source published last month contradicts a page written last year. You can have a perfectly valid, fully conformant OKF bundle that's wrong, or out of date, and the format won't say a word. Being wrong is not a validation error. That's the knot: _a field is not a process._ A timestamp is inert. If you're reaching for a wiki over a vector database, here's the trap. RAG re-derives its answer from the chunks on every query, so its staleness is bounded by what it just retrieved. A wiki's staleness is unbounded: it's whatever the last curation pass left behind, however long ago that was. And a stale wiki page still _reads as authoritative_, so it rots faster than RAG because nobody suspects the well-formatted page. I know this because I run a knowledge base that would have rotted if I hadn't planned for it. ## What `okfctl` Does My own corpus is currently 254 curated concepts, the research and method notes my agents read before they do anything, stored as an OKF bundle. For a while it was a bundle with no process behind it, aging one commit at a time. So I built the check the spec left out. [okfctl](https://okfctl.dev) is an open-source CLI, Apache-2.0, on [GitHub](https://github.com/cwest/okfctl). It's a single static Go binary you install and point at a folder, and the whole thing is designed around one refusal: the core stays pure Go, offline, and model-free, so every command below produces the same answer on your laptop that it produces in my CI. What follows is that binary run against my real 254-node bundle. Every number is its actual output. ### Get It, Then Run It Once The rest of this post shows the tool against a corpus that already exists. You don't have one yet, so start smaller. Install it with the one-liner, which detects your OS and architecture and drops `okfctl` on your `PATH`: ```sh curl -sSL https://okfctl.dev/install.sh | sh ``` On a Mac I use Homebrew instead, `brew install cwest/tap/okfctl`, and if you have a Go toolchain, `go install github.com/cwest/okfctl@latest` builds it from source. Any of the three leaves you with the same binary. Now make a bundle and check it. `bundle init` scaffolds the two reserved files the spec requires; `validate` confirms what it wrote is legal OKF: ```text $ okfctl bundle init ./kb Initialized OKF bundle in ./kb $ okfctl validate ./kb OK: bundle conforms to the OKF spec floor ``` Two commands, and you have a conforming, empty bundle. You now have what the rest of this post is about, and what the OKF spec guarantees. Everything below is how a bundle this clean can still go wrong, and what to run so it doesn't. ### Conformance and Health Are Two Different Questions Start with the most important pair. `validate` asks whether the bundle is a legal OKF bundle. `lint` asks whether it's holding together as a body of knowledge. On my corpus, both come back clean: ```text $ okfctl validate ./bundles/knowledge OK: bundle conforms to the OKF spec floor $ okfctl lint ./bundles/knowledge OK: no lint findings ``` Two green checks, two different floors. `validate` measures the bundle against the OKF specification. `lint` measures orphans, broken cross-links, and coverage gaps, the curation hygiene. A perfectly conformant folder can still be quietly rotting, and only `lint` catches it. ### `analyze`: Make the Rot Visible `lint` gives a pass/fail gate. `analyze` is the microscope. It walks the whole bundle and reports where the knowledge is _weak_, not where it's _malformed_: ```text $ okfctl analyze ./bundles/knowledge # OKF Corpus Analysis 254 node(s), 4123 internal link(s). Stale threshold: 180d. ... ## Coverage gaps — thin nodes (need expansion) - casey/agentic-efficiency.md (9 lines) - design/spacing-rhythm-and-density.md (13 lines) ... ## Structure — near-duplicate slugs (rename/merge candidates) - security/authz/abac.md ≈ security/authz/rbac.md ... 435 actionable signal(s) across all dimensions. ``` Four hundred thirty-five signals on a corpus that just passed both gates. A nine-line concept that pretends to be a concept. Nine nodes with no citation behind them. Two access-control pages, `abac.md` and `rbac.md`, similar enough that one of them is probably swallowing the other's traffic. None of that's a spec violation. `analyze` is the timestamp's missing conscience: it does the noticing the frontmatter field can't. ### `eval`: A Machine Gate on Trust This is the command I'm proudest of. `eval` decomposes a node's trustworthiness along four dimensions borrowed from the TACA framing: Transparency, Accuracy, Calibration, and Alignment. Only one of those can be checked without a model. `eval transparency` is deterministic. It checks that a node's provenance is actually there, that it carries a grade, that its internal citations resolve. Run against my corpus, it found two real problems: ```text $ okfctl eval transparency ./bundles/knowledge grade-vocabulary: design/spacing-rhythm-and-density.md has authority: "DEPRECATED", an off-vocabulary value carried by only 1 node(s) (likely a typo/drift) grade-vocabulary: design/ux-psychology/deceptive-patterns.md has authority: "high", an off-vocabulary value carried by only 1 node(s) (likely a typo/drift) 2 transparency finding(s) ``` Two nodes carrying authority grades that aren't in the vocabulary, `DEPRECATED` and `high`, each the only node in 254 to use its value. That's drift, the kind of slow schema erosion that no one notices until a query trusts the wrong page. okfctl's own help text calls this "the first machine gate that touches trust rather than format," and that's the right claim, precisely because it's a narrow one. The other three dimensions need a model or the network to judge. Instead of guessing whether a claim matches its source, `eval sample` scaffolds an eval-set, one row per node with every extractable field pre-filled and the judgment slots left empty, and hands it to a human or an out-of-band LLM judge to complete. The tool computes no truth verdict it can't verify on its own. ### `search`: Query the Bundle With No Model and No Index Search runs against the bundle with no embedding model and no prebuilt index, which matters if you want a curation check you can run on every commit without a GPU in the loop. Lexical mode matches title, tag, type, or body. The mode I reach for more is `--neighbors`, which walks the link graph instead of the text: ```text $ okfctl search --neighbors research/a-field-is-not-a-process.md ./bundles/knowledge casey/usermd-anchor-restructure-decision.md Decision Memo depth=1 method/node-worthiness-graph-richness.md Research Brief depth=1 research/agentic-harness.md Research Brief depth=1 research/breadth-access-capability-layer.md Concept depth=1 research/core-instruction-files-and-memory-layering.md Research Brief depth=1 research/go-vs-rust-okfctl-cli-spike.md Research Brief depth=1 research/llm-wiki.md Concept depth=1 research/okf-write-time-curation.md Research Brief depth=1 research/rag-vs-write-time-curation.md Concept depth=1 research/semantic-recall-store-design.md Research Brief depth=1 research/the-invisible-moat.md Concept depth=1 ``` The bundle is a graph, so I can ask it graph questions: what sits one hop from this concept? The answer is the neighborhood this very post grew out of, right down to the spike where I argued myself into building the thing in Go. Semantic vector search exists too, but as a separate plugin, deliberately outside the core. ### `migrate`: Two Phases So It Never Learns to Guess When OKF went from v0.1 to v0.2, two keys got renamed. `migrate` upgrades a bundle across that break in two phases. Phase one is pure read: it computes every mechanical edit and enumerates every edit that needs a judgment call, then writes a plan file and touches nothing else. Phase two applies only the mechanical edits, order-preserving and additive, and re-validates. The judgment items, a prose citation with no resource behind it, a rename with no recorded actor, are never guessed. They stay in the plan for a person to resolve. ### `graph export` and `serve`: The Bundle Is a Graph, So Treat It Like One `graph export` hands you the whole link structure as JSON or Graphviz DOT, so the 254 nodes and 4,123 edges become something you can pipe into other tooling. `serve` renders the same graph as an interactive page, assets baked into the binary. A knowledge base is a network of concepts. It can help to visualize it. Here's mine. `graph export` produces the DOT; a short script colors each node and its outbound edges by top-level folder and sizes nodes by degree, then Graphviz lays it out: ```sh $ okfctl graph export --format dot ./bundles/knowledge > kb.dot $ ./scripts/color-graph.py kb.dot | sfdp -Tpng > graph.png ``` ![The 254 nodes and 4,123 edges of my knowledge base, laid out by force-directed graph, colored by top-level folder. Labels are stripped.](/images/posts/a-field-is-not-a-process/knowledge-graph.jpg) Every dot is a concept (node) and every line is a link (edge). That picture is also the argument: nothing in the folder format keeps a network that dense honest as it grows, which is what the rest of these commands are for. ### One More Refusal: Plugins Everything above is core, and core is pure Go and offline on purpose. The things that need a model or the network, semantic search, an HTTP API, live outside as `okfctl-` plugins that the CLI discovers on your `PATH`. You can also write your own plugins. ## Wire It In, or It Won't Run A command you have to remember to run is a command that eventually doesn't. So the check has to sit inside the loop: ```mermaid flowchart LR A[Write or edit a concept] --> B[Commit] B --> C{"okfctl lint --strict"} C -->|"orphan · stale link · contradiction"| D[Fail the build] D --> A C -->|clean| E[Merge] E --> F[Corpus stays true] F --> A ``` I run it at two points. A pre-commit hook stops a bad concept before it lands, and CI runs it as a hard gate that fails the whole build on any curation error. The `modified` field is still inert. The hook and the gate are what make it mean something, because now something _runs_ when the folder changes. There's a measurable payoff, not just a hygienic one. When I moved the corpus to passage-level search indexing, so a hit points at the paragraph instead of the file, retrieval quality on the real corpus went from 0.545 to 0.909. A maintained bundle earns better answers than a pile of valid-but-unkept files. ## Choose the Whole Idea, Not Half of It If you're choosing a wiki over RAG for your agents because you read the same gist I did, choose the whole idea. The folder is the part you get for free. The recurring, build-failing check that keeps it true is the part you have to build, or borrow, or you won't have the thing you actually want. OKF is a good container. Use it, then add the tools and process so your information doesn't go stale. [okfctl](https://okfctl.dev) is one static binary, and it's the difference between a knowledge base and a graveyard that validates. If you want to borrow the tool instead of building your own, that's the one command: ```sh curl -sSL https://okfctl.dev/install.sh | sh ``` Then `okfctl bundle init ./kb` and you're where I was when I started. --- It's a familiar story for many technologists: the personal website, once a point of pride, gradually becomes a relic. Mine was no different – a trusty [Jekyll](https://jekyllrb.com/) site, faithfully serving content for over a decade, but increasingly feeling like a digital time capsule. The build process felt clunky, the development environment lagged behind modern practices, and the desire for a refresh was strong. But who has the time for a full rewrite? That's where things got interesting. I decided to embrace the wave of AI advancements and see if I could partner with AI, specifically Google's Gemini models, to not only migrate my site to the modern [Astro](https://astro.build/) framework but to do it _fast_. My goal: a complete migration, a new local dev setup, and a fully automated CI/CD pipeline, all within about three days. Spoiler alert: We did it. And it was a _blast_. ## Phase 1: AI as the Architect – Crafting the Blueprint Any successful project needs a plan. Instead of spending days researching Astro best practices, migration gotchas, and optimal configurations for my specific setup (Mac Studio M2 Ultra, GitHub Pages), I turned to AI. ### Step 1: The Comprehensive Guide My first move was to leverage the deep research capabilities of Gemini 2.5 Pro. I went to [gemini.google.com](https://gemini.google.com/) and used the following prompt, asking it to take on the role of generating a detailed migration guide: ```markdown wrap # Project: Migrate Personal Site/Blog from Jekyll to Astro ## Goal Migrate my personal site/blog from Jekyll to Astro, set up a local development environment, and establish a continuous delivery system. ## Project Details - **Source Code Management:** `git` - **Repository Host:** GitHub - **GitHub Username:** `cwest` - **Repository Name:** `cwest.github.io` - **Publishing Platform:** GitHub Pages - **Custom Domains:** `caseywest.com`, `geeknest.com` (configured via `CNAME`) - **URL Structure:** Blog posts hosted at root level (`/[slug]`). - **Content Type:** Technical content, including code snippets and examples. - **Code Highlighting Requirement:** Use a high-quality syntax highlighter with user-friendly features for code examples. ## Deployment Workflow 1. Work on features/posts in a topic branch locally. 2. Push the topic branch to GitHub and create a Pull Request (PR). 3. The PR should trigger automated checks (tests, linters, etc.). 4. Upon merging the PR into the `main` branch, the site should be automatically built and deployed to GitHub Pages. ## Development Environment Setup (Local Machine: Mac Studio M2 Ultra) - **IDE:** VS Code - **Shell:** `zsh` - **Terminal:** WezTerm - **Web Browser:** Chrome - **Node.js Version Management:** Need a solution (e.g., `nvm`, `fnm`). - **VS Code Extensions:** Required for efficient development with: - Astro (`.astro` files) - Strict TypeScript (`.ts`, `.tsx`) - CSS (including SASS/SCSS - `.css`, `.sass`, `.scss`) - SVG (`.svg`) - HTML (`.html`) - Markdown (`.md`) - Vibe Coding with Roo Code - Google’s Gemini 2.5 Pro model - **Additional Tools:** Open to suggestions for other VS Code extensions, command-line tools, automations, etc., to enhance productivity. ## Continuous Delivery (CD) - **Platform:** GitHub Actions - **Target:** Deploy the built Astro site to GitHub Pages. ## Daily Workflow Example Request Provide a step-by-step guide for a typical daily workflow: 1. Creating a new blog post. 2. Writing content locally with live preview. 3. Publishing the content (following the defined deployment workflow). 4. Resetting the local environment for the next task. ## Request for AI Research and provide a detailed, step-by-step guide for setting up this modern development environment and workflow on my specified Mac Studio M2 Ultra. The guide should aim to be actionable within a few hours. ``` The result? A remarkably thorough, step-by-step guide (the '[Astro Migration and Deployment Guide](https://docs.google.com/document/d/1sI5VKjLN-JY1qVb2b2kyGAHWvETqwpi_RVwoLqcgWJg/edit?usp=sharing)' attached to this project!) covering everything from Node.js version management (comparing FNM and Volta) to VS Code extension recommendations, Astro configuration nuances, content migration strategies (hello, Content Collections!), and even detailed GitHub Actions workflow YAML for CI/CD. It felt like having a senior architect draft a personalized implementation manual just for me. ### Step 2: The Actionable Project Plan With the 'what' and 'how' documented in the guide, I needed a 'to-do' list. I took the generated guide and headed over to [Google AI Studio](https://aistudio.google.com/). There, I used Gemini 2.5 Pro with Google Search enabled (to ensure up-to-date accuracy) and provided the _entire guide_ as context along with this prompt: ```markdown wrap Based on this research and guide, create a detailed step-by-step project plan broken down into Goals comprised of one or more Tasks. In your output structure prefer headings to delineate Goals and Tasks instead of multi-level bullet point lists or simple bold text. I will use this project plan to execute the steps necessary to create my new blogging development environment, deployment process, and writing process from scratch. If necessary to write a detailed, up to date, and accurate task use Google Search to do further research and/or validate your plan. ``` Gemini returned a beautifully structured project plan (the '[Astro Migration and Deployment Project Plan](https://docs.google.com/document/d/1nh_oLJRjYP3Xt7xwWKPm5mfCQOGqyNcOIwHOahul2_A/edit?usp=sharing)'!), complete with task breakdowns, specific commands, and even `[]` placeholders for checkmarks. It transformed the comprehensive guide into an executable sequence, ready for me to tackle. This AI-generated plan became my roadmap for the next couple of days. ## Phase 2: AI as the Coding Companion – "Vibe Coding" the Migration and Beyond With a solid plan in hand, it was time for execution. This is where the real fun began, pairing my development work with an AI coding assistant directly within VS Code. I configured the fantastic [Roo Code](https://marketplace.visualstudio.com/items?itemName=RooVetGit.Roo-Code) extension (shoutout to the RooVetGit team!) to use the `gemini-2.5-pro-preview-03-25` model directly via its API. I also set Roo Code to auto-approve reading files and retrying actions, streamlining the interaction. My workflow settled into a rhythm I call "Vibe Coding": 1. **Consult the Plan:** Pick the next task from the AI-generated project plan. 2. **Execute Simple Tasks:** If it was a straightforward command (`brew install fnm`, `git checkout -b ...`), I'd run it myself in the terminal. 3. **Delegate Complex Tasks to Roo:** For anything requiring code generation, file manipulation, refactoring, or content conversion, I'd prompt Roo Code. The key was treating the AI like a pair programmer. I provided context, clear instructions, and source material (like old Jekyll posts or existing code), and let it handle the heavy lifting or the first draft. **Example: Content Conversion** Migrating old Jekyll posts required converting frontmatter and ensuring path compatibility with Astro's content collections. I'd give Roo Code a prompt like this: ```text I want to integrate this old Jekyll post into my new site as a content piece. The end-point URL must be `/your-software-is-made-of-people`. Below is the original Jekyll source in Markdown. Convert it into a Markdown source for Astro and my site in this project. Use the attached image as the hero image. Follow conventions to name the hero image with the same filename base as the post. Here's the post: --- # ... (Jekyll frontmatter and content) ... --- ``` Roo, powered by Gemini, would read the original file, understand the context of my Astro project (thanks to its file-reading capability), parse the request, and generate the new Markdown file with updated frontmatter (including the `slug` field based on my URL requirement) and correctly formatted content. **Example: Code Refactoring** AI isn't just for generating new code; it's fantastic for improving existing code. After getting some initial functionality working, like a script to generate hero images (more on that later!), I used Roo Code to elevate the quality: ```markdown wrap Refactor @/scripts/generate-hero-image.ts like an expert TypeScript programmer who cares about writing beautiful code. Consider the following: - The highest standards of idiomatic programming - Don't Repeat Yourself (DRY) - No useless comments - Self documenting code - Best practices in TypeScript for documeting functions, interfaces, etc - Easy to maintain code that's not complex or clever unless it absolutely must be - Extremely readable code that new TypeScript programmers would be able to understand ``` The results were consistently impressive, transforming functional code into cleaner, more maintainable, and idiomatic TypeScript. **Beyond the Plan: AI-Assisted Tooling** What truly highlighted the power of this "Vibe Coding" approach was how quickly I could build _new_ tooling that wasn't even in the original migration plan. The AI wasn't just helping me execute predefined steps; it was enabling rapid development to improve my ongoing workflow. **Tool 1: Scaffolding New Posts (`new-post.ts`)** I realized I needed a quick way to create new blog post files with the correct frontmatter structure. Instead of manually copying and pasting, I prompted Roo Code to help build a simple Node.js script using [TypeScript](https://www.typescriptlang.org/) and libraries like [`fs/promises`](https://nodejs.org/api/fs.html#promises-api) and [`gray-matter`](https://github.com/jonschlinkert/gray-matter). The resulting `scripts/new-post.ts` script takes a title string, automatically generates a URL-friendly slug, creates the Markdown file (e.g., `src/content/post/a-great-new-post.md`), and pre-fills the frontmatter with the title, a placeholder description, the current date, and a placeholder for the hero image path. ```typescript title="new-post.ts" collapse={1-36,69-96} import fs from 'fs/promises' // Use promises API for async operations import path from 'path' import { exit } from 'process' // Explicit import for exit import matter from 'gray-matter' // Re-import gray-matter /** * Defines the structure for the front matter of a blog post. */ interface FrontMatter { title: string description: string pubDate: string // ISO 8601 date string heroImage: string } /** * Generates a URL-friendly slug from a given text string. * Converts to lowercase, replaces spaces with hyphens, removes non-alphanumeric characters (except hyphens), * and trims leading/trailing hyphens. * * @param text - The input string to slugify. * @returns The generated slug string. */ function slugify(text: string): string { return text .toString() // Ensure input is a string .toLowerCase() .trim() .replace(/\s+/g, '-') // Replace spaces and consecutive whitespace with a single hyphen .replace(/[^\w-]+/g, '') // Remove characters that are not word characters, digits, or hyphens .replace(/--+/g, '-') // Collapse multiple consecutive hyphens into one .replace(/^-+/, '') // Remove hyphens from the start .replace(/-+$/, '') // Remove hyphens from the end } /** * Creates a new markdown file for a blog post with predefined front matter. * * @param title - The title of the new blog post, provided as a command-line argument. */ async function createNewPost(title: string): Promise { const slug = slugify(title) const filename = `${slug}.md` const targetDir = path.join('src', 'content', 'post') const filePath = path.join(targetDir, filename) // Prepare front matter data const frontMatterData: FrontMatter = { title: title, // Let gray-matter handle quoting description: '# Add a brief description here', pubDate: new Date().toISOString(), heroImage: `# Add path to hero image, e.g., /images/posts/${slug}/hero.jpg`, } // Define the initial content body const mainContent = 'Write your post content here...' // Combine front matter and content using gray-matter const fileContent = matter.stringify(mainContent, frontMatterData) // Use default stringify try { // Ensure the target directory exists before writing the file await fs.mkdir(targetDir, { recursive: true }) // Write the file, using 'wx' flag to prevent overwriting existing files await fs.writeFile(filePath, fileContent, { flag: 'wx' }) console.log(`✅ Created new post: ${filePath}`) } catch (error) { // Type guard to check if the error is a Node.js file system error if (error instanceof Error && 'code' in error) { if (error.code === 'EEXIST') { console.error(`❌ Error: File already exists at ${filePath}`) } else { console.error(`❌ Error creating file: ${error.message} (Code: ${error.code})`) } } else { // Handle unexpected error types console.error('❌ An unexpected error occurred:', error) } exit(1) // Exit with error code } } // --- Script Execution --- // Get the post title from command line arguments const postTitle = process.argv[2] // Validate input if (!postTitle) { console.error('❌ Error: Please provide a post title as the first argument.') console.log('Usage: pnpm new-post "Your Post Title"') exit(1) // Exit with error code } // Run the main function createNewPost(postTitle).catch((err) => { // Catch any unhandled promise rejections from createNewPost console.error('❌ An unexpected error occurred during script execution:', err) exit(1) }) ``` I added it to my `package.json`: ```json "scripts": { // ... other scripts "new-post": "tsx scripts/new-post.ts" } ``` Now, starting a new post is as simple as: ```shell pnpm run new-post "A Great New Post Title" ``` **Tool 2: Generating Hero Images (`generate-hero-image.ts`)** Then came the _really_ cool part. I wanted unique, AI-generated hero images for each post. This was a more complex task involving external API calls, file handling, and updating existing files. Again, I collaborated with Roo Code, iterating on prompts to build `scripts/generate-hero-image.ts`. This script does several things: 1. Takes a post slug and a text prompt as arguments. 2. Uses the [`@google-cloud/aiplatform`](https://github.com/googleapis/google-cloud-node/tree/main/packages/google-cloud-aiplatform) Node.js client library to call the [Google Cloud Vertex AI Prediction API](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions). 3. Specifically targets the `imagegeneration@006` model endpoint, which provides access to Google's incredibly powerful [Imagen 3](https://cloud.google.com/vertex-ai/docs/generative-ai/image/overview) text-to-image model. (Seriously, the quality is phenomenal!) 4. Receives the generated image data (as base64). 5. Saves the image to the correct directory (e.g., `public/images/posts/a-great-new-post/hero.jpg`). 6. Parses the corresponding Markdown post file using `gray-matter` and updates the `heroImage` frontmatter field with the correct path to the newly saved image. ```typescript title="generate-hero-image.ts" collapse={1-179,235-358} /* eslint-disable no-console */ // Allow console logs for CLI script feedback import * as fs from 'fs/promises' import * as path from 'path' import { PredictionServiceClient, helpers } from '@google-cloud/aiplatform' import { status as GrpcStatus } from '@grpc/grpc-js' // Rename for clarity import dotenv from 'dotenv' import matter from 'gray-matter' // Used for frontmatter parsing import type { protos } from '@google-cloud/aiplatform' // --- Type Aliases & Interfaces --- // Specific protos types for request/response clarity type IPredictRequest = protos.google.cloud.aiplatform.v1.IPredictRequest type IPredictResponse = protos.google.cloud.aiplatform.v1.IPredictResponse /** Defines the structure for script configuration settings. */ interface ScriptConfig { readonly postsDir: string readonly imageOutputDirRoot: string readonly imagePublicPathRoot: string readonly googleProjectId: string readonly googleLocation: string readonly googleAiModel: string readonly heroImageFilename: string readonly defaultAspectRatio: string } /** Defines the structure for parsed command-line arguments. */ interface CliArguments { readonly postSlug: string readonly userPrompt: string } /** Represents the result of parsing a Markdown file with frontmatter. */ type ParsedPost = matter.GrayMatterFile // --- Constants --- const DEFAULT_GOOGLE_LOCATION = 'us-central1' const DEFAULT_HERO_FILENAME = 'hero.jpg' const DEFAULT_ASPECT_RATIO = '16:9' const GOOGLE_AI_MODEL = 'imagegeneration@006' // Imagen model identifier const ADC_ERROR_MESSAGE = 'Could not load the default credentials' const ADC_HELP_MESSAGE = ` ❌ Authentication Error: Ensure Application Default Credentials (ADC) are valid. Run: gcloud auth application-default login ` // --- Configuration Loading & Validation --- /** * Loads configuration from environment variables and defaults. * @returns The script configuration object. * @throws {Error} If essential configuration (GOOGLE_PROJECT_ID) is missing. */ function loadConfig(): ScriptConfig { dotenv.config() // Load .env file const googleProjectId = process.env.GOOGLE_PROJECT_ID if (!googleProjectId) { console.error( 'Error: GOOGLE_PROJECT_ID environment variable is not set.', 'Please ensure it is defined in your .env file or environment.', ) process.exit(1) // Exit early if critical config is missing } return Object.freeze({ // Make config immutable postsDir: path.resolve(process.cwd(), 'src/content/post'), imageOutputDirRoot: path.resolve(process.cwd(), 'public/images/posts'), imagePublicPathRoot: '/images/posts', googleProjectId, googleLocation: process.env.GOOGLE_LOCATION || DEFAULT_GOOGLE_LOCATION, googleAiModel: GOOGLE_AI_MODEL, heroImageFilename: DEFAULT_HERO_FILENAME, defaultAspectRatio: DEFAULT_ASPECT_RATIO, }) } // --- Utility Functions --- /** * Checks if a file exists at the given path. * @param filePath - The path to the file. * @returns True if the file exists, false otherwise. */ async function fileExists(filePath: string): Promise { try { await fs.access(filePath) return true } catch { return false } } /** * Parses command-line arguments for post slug and prompt. * Exits the process with an error message if arguments are invalid. * @returns An object containing the postSlug and userPrompt. */ function parseArguments(): CliArguments { const args = process.argv.slice(2) // Skip node executable and script path if (args.length !== 2 || !args[0]?.trim() || !args[1]?.trim()) { console.error('Usage: pnpm run generate-hero-image ""') console.error('Example: pnpm run generate-hero-image my-new-post "A futuristic cityscape"') process.exit(1) } return Object.freeze({ // Make args immutable postSlug: args[0], userPrompt: args[1], }) } /** * Handles common script errors, logs informative messages, and exits. * @param error - The error object or message. * @param context - Optional context message (e.g., "while generating image"). */ function handleError(error: unknown, context?: string): never { // 'never' indicates function exits console.error(`\n❌ An error occurred${context ? ` ${context}` : ''}:`) if (error instanceof Error) { console.error(` Message: ${error.message}`) // Check for specific known error types or messages if (error.message.includes(ADC_ERROR_MESSAGE)) { console.error(ADC_HELP_MESSAGE) } else { // Check for gRPC status codes if available (often indicates API issues) const grpcError = error as any // Use 'any' cautiously for type casting if (grpcError && typeof grpcError.code === 'number') { console.error(` gRPC Code: ${grpcError.code} (${GrpcStatus[grpcError.code] || 'Unknown'})`) if (grpcError.details) console.error(` Details: ${grpcError.details}`) if (grpcError.code === GrpcStatus.UNAUTHENTICATED || grpcError.code === GrpcStatus.PERMISSION_DENIED) { console.error(ADC_HELP_MESSAGE) } } else if (error.stack) { // Provide stack trace for other errors if available console.error(` Stack: ${error.stack}`) } } } else { // Handle non-Error types console.error(' Error details:', error) } process.exit(1) } // --- Core Logic Functions --- /** * Reads a post file and parses its frontmatter. * @param postFilePath - Path to the markdown post file. * @returns The parsed GrayMatterFile object. * @throws {Error} If the file doesn't exist or cannot be parsed. */ async function readAndParsePost(postFilePath: string): Promise { if (!(await fileExists(postFilePath))) { throw new Error(`Post file not found at ${postFilePath}`) } try { const postFileContent = await fs.readFile(postFilePath, 'utf-8') return matter(postFileContent) } catch (error) { throw new Error( `Failed to read or parse post file ${postFilePath}: ${error instanceof Error ? error.message : String(error)}`, ) } } /** * Builds the prediction request object for the Google AI Platform. * @param prompt - The text prompt for image generation. * @param config - The script configuration. * @returns The constructed IPredictRequest object. * @throws {Error} If prompt or parameters cannot be converted. */ function buildPredictRequest(prompt: string, config: ScriptConfig): IPredictRequest { const endpoint = `projects/${config.googleProjectId}/locations/${config.googleLocation}/publishers/google/models/${config.googleAiModel}` const instanceValue = helpers.toValue({ prompt }) if (!instanceValue) { throw new Error('Failed to convert prompt instance to IValue') } const parametersObj = { sampleCount: 1, aspectRatio: config.defaultAspectRatio, // Add other parameters like negativePrompt, seed, etc., here if needed } const parametersValue = helpers.toValue(parametersObj) if (!parametersValue) { throw new Error('Failed to convert parameters object to IValue') } return { endpoint, instances: [instanceValue], parameters: parametersValue, } } /** * Calls the Google AI Platform Prediction Service. * @param request - The prediction request object. * @param modelName - The name of the AI model being used (for logging). * @returns The prediction response object. * @throws {Error} If the API call fails. */ async function callPredictionService(request: IPredictRequest, modelName: string): Promise { console.log(`\nSending request to AI Platform Prediction Service (Model: ${modelName})...`) // Instantiate the client just before the call const predictionServiceClient = new PredictionServiceClient() // The predict method returns a tuple: [response, request, options] const [response] = await predictionServiceClient.predict(request) if (!response) { // This case should ideally be handled by the SDK throwing an error, // but adding a check for robustness. throw new Error('Received undefined response from AI Platform predict call.') } console.log('Received response from AI Platform.') return response } /** * Extracts the base64 encoded image data from the prediction response. * @param response - The prediction response object. * @returns Base64 encoded string of the generated image. * @throws {Error} If the response structure is unexpected or lacks image data. */ function extractImageData(response: IPredictResponse): string { // Safely access the prediction data using optional chaining const imageBase64 = response.predictions?.[0]?.structValue?.fields?.bytesBase64Encoded?.stringValue if (!imageBase64) { console.error('Unexpected response structure:', JSON.stringify(response, null, 2)) throw new Error('No valid base64 image data found in the AI Platform response.') } console.log('Image data extracted successfully.') return imageBase64 } /** * Generates an image using the Google AI Platform Prediction Service. * Orchestrates request building, API call, and response parsing. * @param prompt - The text prompt for image generation. * @param config - The script configuration. * @returns Base64 encoded string of the generated image. */ async function generateImage(prompt: string, config: ScriptConfig): Promise { try { const request = buildPredictRequest(prompt, config) const response = await callPredictionService(request, config.googleAiModel) const imageBase64 = extractImageData(response) return imageBase64 } catch (error) { // Use the centralized error handler handleError(error, 'while generating image') } } /** * Saves the generated image to the specified path. * Creates the output directory if it doesn't exist. * @param imageBase64 - Base64 encoded image data. * @param outputPath - The full path where the image should be saved. * @param outputDir - The directory where the image will be saved. */ async function saveImage(imageBase64: string, outputPath: string, outputDir: string): Promise { try { await fs.mkdir(outputDir, { recursive: true }) await fs.writeFile(outputPath, imageBase64, 'base64') console.log(`Image saved to: ${outputPath}`) } catch (error) { // Use the centralized error handler for saving errors handleError(error, `while saving image to ${outputPath}`) } } /** * Updates the post's frontmatter with the new hero image path. * @param postFilePath - Path to the markdown post file. * @param parsedPost - The parsed frontmatter and content. * @param imagePublicPath - The public URL path for the hero image. */ async function updateFrontmatter(postFilePath: string, parsedPost: ParsedPost, imagePublicPath: string): Promise { try { // Create a mutable copy of data for modification const updatedData = { ...parsedPost.data, heroImage: imagePublicPath } const updatedPostFileContent = matter.stringify(parsedPost.content, updatedData) await fs.writeFile(postFilePath, updatedPostFileContent, 'utf-8') console.log(`Updated frontmatter in: ${postFilePath}`) } catch (error) { // Log a warning instead of exiting, as the image was generated. console.warn(`\n⚠️ Warning: Failed to update frontmatter for ${postFilePath}.`) console.warn(` Error: ${error instanceof Error ? error.message : String(error)}`) console.warn(' The image was generated and saved, but the post file needs manual updating.') } } // --- Main Execution --- /** * Main function to orchestrate the hero image generation process. */ async function main() { const config = loadConfig() // Load and validate config first const { postSlug, userPrompt } = parseArguments() console.log(`Generating hero image for post: ${postSlug}`) console.log(`User prompt: "${userPrompt}"`) // Construct paths using the validated config const postFilePath = path.join(config.postsDir, `${postSlug}.md`) const imageOutputDir = path.join(config.imageOutputDirRoot, postSlug) const imageOutputPath = path.join(imageOutputDir, config.heroImageFilename) // Ensure consistent path separators for URLs const imagePublicPath = [config.imagePublicPathRoot, postSlug, config.heroImageFilename] .join('/') .replace(/\/+/g, '/') // Normalize slashes // 1. Read and parse the post file const parsedPost = await readAndParsePost(postFilePath).catch((error) => handleError(error, 'while reading post file'), ) // Note: Add logic here if you want to enhance the prompt with post data const fullPrompt = userPrompt // Keep it simple for now // 2. Generate the image via AI Platform const imageBase64 = await generateImage(fullPrompt, config) // Error handled within generateImage // 3. Save the generated image await saveImage(imageBase64, imageOutputPath, imageOutputDir) // Error handled within saveImage // 4. Update the post's frontmatter await updateFrontmatter(postFilePath, parsedPost, imagePublicPath) // Logs warning on failure console.log(`\n✅ Successfully generated and linked hero image: ${imagePublicPath}`) } // --- Script Entry Point --- main().catch((error) => { // Catch any unexpected errors not handled by specific try/catch blocks or handleError handleError(error, 'during script execution') }) ``` I added this script to `package.json` too: ```json "scripts": { // ... other scripts "generate-hero-image": "tsx scripts/generate-hero-image.ts" } ``` Now, generating a hero image is a single command: ```shell wrap pnpm run generate-hero-image a-great-new-post "Hyperrealistic apple on a park bench in the sunset" ``` ![A red apple sitting on a wooden park bench during sunset](/images/posts/ai-driven-development-modernizing-a-decade-old-website-in-3-days/apple-on-bench.png) Building these helper scripts manually might have taken significant time, potentially derailing the core migration. With AI assistance, they became quick, iterative additions that substantially improved the final workflow. This iterative "vibe coding" process – human direction, AI execution/drafting, human review/refinement – allowed me to move through the project plan _and beyond_ at an incredible pace. ## The Result: A Modern Foundation in Record Time In roughly three days of focused effort (interspersed with regular life, of course!), the migration was complete: - **Modern Astro Site:** The old Jekyll blog was reborn as a performant Astro site. - **Streamlined Dev Environment:** A clean setup using [FNM](https://github.com/Schniz/fnm) for Node version management, VS Code dialed in with essential extensions like [Astro Language Support](https://marketplace.visualstudio.com/items?itemName=astro-build.astro-vscode), [Prettier](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode), [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint), and crucially, Roo Code connected to Gemini. - **Automated CI/CD:** A robust [GitHub Actions](https://github.com/features/actions) workflow automatically linting, type-checking, building, and deploying the site to [GitHub Pages](https://pages.github.com/) on every merge to `main`. - **Custom Domain:** Correctly configured DNS for `caseywest.com` pointing to the GitHub Pages instance. - **New Workflow:** A defined, efficient process for writing and publishing new content, enhanced by custom AI-generated tooling. Doing this manually likely would have been a multi-week project, minimum. The AI partnership compressed that timeline dramatically. ## Reflections and The Road Ahead This experience solidified my belief that AI is fundamentally changing software development. It's not about replacing developers but augmenting them, acting as an incredibly powerful coding companion, architect, and assistant. - **Prompting is Key:** The quality of the AI's output (both the initial guide/plan and the code/content generated during vibe coding) was directly proportional to the clarity and detail of my prompts. - **AI Accelerates, You Steer:** AI handled the tedious, the repetitive, and the initial drafts, freeing me up to focus on the strategic decisions, the tricky integrations, and the final polish. - **Iterative Collaboration:** The back-and-forth felt natural, like working with a very fast, knowledgeable, tireless collaborator. The journey doesn't end here. The next steps involve further exploring Astro's features, refining the site's design, and, of course, writing more content using this new, supercharged workflow. ## Your Turn! Feeling inspired? Have an old project gathering digital dust? I highly encourage you to try replicating my process: 1. **Define Your Goal:** What do you want to modernize or build? Be specific about your current setup and desired outcome. 2. **Generate Your Guide:** Use a powerful model with deep research capabilities (like Gemini 2.5 Pro via the [Gemini website](https://gemini.google.com/)) and provide it with a detailed prompt outlining your project, environment, and requirements, asking it to create a comprehensive guide. 3. **Create Your Action Plan:** Take the guide generated in the previous step and use it as context in a tool like [Google AI Studio](https://aistudio.google.com/). Use a model with search capabilities (like Gemini 2.5 Pro with Google Search) and prompt it to create a step-by-step project plan based _on that specific guide_. 4. **Try "Vibe Coding":** Set up an AI assistant like Roo Code in your editor, connect it to your preferred model (like Gemini), and start executing your project plan. Delegate coding, conversion, and refactoring tasks to your AI partner. Don't be afraid to go beyond the plan and build new helper tools along the way! The future of development is collaborative, and AI is ready to be your coding companion. Give it a try – you might be surprised how quickly you can bring your ideas to life! --- > **Note:** This post was originally written in May 2015. While many core concepts remain relevant, the specific tools and landscape of remote work have evolved significantly. This content is presented as originally written for posterity. It's not inherently harder to be a tech lead while remote or on a distributed team – it's more deliberate. I [tweeted](https://twitter.com/caseywest/status/597027263796912128) this sentiment recently along with the concept of _durable communication_. That phrase is inspired by data storage which is typically grouped into two categories: durable and ephemeral. I often hear people say it's _obviously_ harder to work remotely, and very hard to be in a leadership position remotely. I think that's a cop out. I don't buy it for a second. _**Before we continue:** This post is about being a remote worker or working on a distributed team. I am convinced this is the path software organizations are rightly headed, and I'm determined to help you get there. If your organization has no remote workers, by choice or otherwise, or if you think remote work or remote leadership is out of the question this post may not be for you. That's cool. On the other hand, maybe it'll give you the tools, vocabulary, and confidence to give it a shot. Either way I look forward to hearing about it._ # What is Durable Communication? Collaboration using deliberately inclusive, transparent, and reliable tools and techniques. It's the cornerstone of successful remote work. # Why do I care? I currently lead three teams in a distributed engineering organization. One is entirely based in California, another entirely in Pennsylvania, and the third is split between those locations. If I'm going to be successful as a leader it's critical that we that we get communication right. I admit to being a change agent and this is one of the ways I've changed the engineering organization. This essay describes how my teams communicate. # Effective Communication Saves Time Communication is more deliberate and intentional when you're remote and communicating effectively. In a distributed team where pockets of people are co-located it's easy to unintentionally make decisions and share facts only with team members who are in the same physical location. It happens in the kitchen or at lunch. When communicating with someone far away from you, physically, you have to choose to share. It's not an accident. If you want to succeed at remote work it requires deliberate changes to the way your team communicates. It's a conscious choice to alter behaviors and culture to better suit a distributed group of people. This is a skill which can be acquired. It takes practice. Here's the thing: it's no more effort than accidental communication and, in fact, it saves everyone time and energy in the long run when information is freely flowing in channels designed to share broadly. Over time it becomes more comfortable to communicate in a shared space and you'll soon find yourself choosing that by default. You'll say, _"I just learned something weird about our data schema. I'll tell you in our chat room."_ As you describe the oddity that is your data model, in the chat room, you're sharing with everyone regardless of physical location. # Transparent by Default One of the most striking changes a team experiences when practicing durable communication is the substantial increase in their transparency. Most communication is in the open. It's archived and indexed. Anyone can read it. It's like giving your team the super power of time travel. In three years someone is going to be asking, _"Why the hell did we define an enum for this obviously boolean attribute and by the way querying it is timing out on large data sets?!"_ They're going to search your chat archives, read commit history, find tickets, and learn that the attribute was supposed to have five values and was simplified three days before launch because otherwise we weren't going to finish[^yagni]. Thank your lucky stars! All the engineers who worked on that feature have moved on to new opportunities and there's nobody left to ask in the hallway. Good thing your communication was durable. Always err on the side of transparency. It'll save your future ass. # Shared Responsibility Responsibility for durable communication is shared among everyone in a distributed organization; everyone is accountable for their part in making collaboration maximally effective.
Your organization
has a responsibility to provide a collection of tools to enable distributed collaboration.
Your team
has a collective responsibility to cultivate a culture built on the techniques of durable communication.
You

have a responsibility to provide continual feedback when communication is becoming ineffective.

This may be during a conference call when two people are having a side conversation and suddenly you can't follow anything being said. "Excuse me, I can't follow multiple conversations at once. Can we do one at a time?"

It may be during a retrospective to highlight key decisions made beyond your visibility. "I was surprised to learn we decided on Elixir for the spike. If it was written down I didn't see it."

# Change _is_ Hard If your organization is expanding from co-located to distributed it will come with growing pains. Once durable communication has been fully embraced it will feel effortless. Transition won't feel that way, and that's okay. Change requires additional effort and _that_ is hard, but it's effort spent going in the right direction so it's well spent. What follows are concrete methods for reducing the pain of distributed work. It's tailored for teams building software. # Tools I'm going to list a handful of options ranging from completely free to paid. These are things I know about. They aren't endorsements unless explicitly stated. In other words, you should do the cost/benefit analysis for your organization and choose wisely. ## Text Chat This is the primary device for durable communication. In my experience it's the most critical tool in your collaboration toolbox. There are some specific features which make it invaluable, and any deployment should have them one way or another:
Archive
An indexed, searchable archive is particularly important. This is a history of your teams' informal communication and, just like in the hallway, a ton of knowledge sharing and key decisions are made here. You want to remember that.
Group Chat
It's not enough to have one-on-one chat capability, however…
Private Chat
it's also important for individuals to be able to communicate.
Full Participation
Everyone in your organization should be able to use it, and should be using it.
Try [HipChat](https://www.hipchat.com/), [Slack](https://slack.com/), [IRC](http://www.irc.org/), [Google Hangouts](http://www.google.com/hangouts/), or [jabber](http://www.jabber.org/). ### ChatOps [ChatOps](https://speakerdeck.com/jnewland/chatops-at-github) is the use of text chat tools for the transparent automation of business practices. It's so awesome it has a heading. Typically ChatOps is achieved by deploying a bot, which connects to your chat system, and can be programmed to do tasks for you. For example, it could build a new release, do a full deployment, pick a place for lunch, or find the perfect animated GIF for the conversation. It's fun and useful! ChatOps also helps introduce context into your conversations. It can be hard to know what's going on when you're on a distributed team and we can bridge that gap by doing some of that work right in a chat room where everyone can see it. No more wondering when a release is happening. You just saw someone start a release, and you just saw your bot report the release happened successfully. Try [hubot](https://hubot.github.com/) or [Lita](https://www.lita.io/). [HipChat](https://www.hipchat.com/) and [Slack](https://slack.com/) also have a lot of built-in integrations with external services to provide the reporting aspects of ChatOps out of the box. More on ChatOps: [Carol Nichols](https://twitter.com/carols10cents), ["How is ChatOps Formed?"](http://slides.com/carolnichols/chatops#/) ([video](https://www.youtube.com/watch?v=BX-iT5xBaMY)). ## Audio/Video Communication Everyone should be able to have a phone call, or Internet equivalent, when they're at work. Voice is fine, usually, and video is higher bandwidth. Non-verbal communication is a huge part of human communication and it should absolutely be a part of your remote work environment. (Please don't make the joke about not wearing pants. It's not funny anymore.) This tool requires some hardware. It's necessary equipment to effectively do your job and your company should provide it or reimburse you for reasonable expenses.
Headphones

Yes, you need headphones. Unless you're sharing audio with a room you should be wearing headphones on a call or video chat. If you don't you run the risk of an audio feedback loop. Software is still spotty at avoiding this.

The other reason for headphones is if you're working on a distributed team with pockets of co-located people there's a likelihood you'll end up on the same call as several people around you. There's nothing worse than hearing someone speak twice, on a slight delay, so it sounds like an echo. Oh, there is something worse: hearing yourself when you're talking, on a slight delay.

Microphone

I recommend getting a set of headphones with a built-in microphone. A lot of the reasons for having one are the same as above.

The times when this breaks down are meetings where most people are co-located and a lot of them are speaking. An example is standup with a couple remote workers and everyone else physically gathered. For these cases I recommend a microphone which can be passed around when someone is speaking. This serves two purposes: high quality audio for the speaker regardless of their location in the room, and it tends to force the group to speak one at a time.

Camera

Most laptops and all-in-one desktops come with cameras these days and they're usually good enough for these purposes.

The times when this breaks down is conference room meetings, often with whiteboards, where a small number of participants are remote. If your company is fancy they can buy great solutions for this problem. A cheap option is an external camera which someone in the meeting can pan and zoom for you. Cheap and effective.

There are three important features of any effective audio/video communication tool: the ability to have one-on-one calls, the ability to conduct group conference calls, and the ability to do either in 60 seconds or less. This collection of features isn't always available in a single tool, and that's okay. Use a few if you have to. It'll pay off. Try [HipChat](https://www.hipchat.com/) (one on one), [Skype](http://skype.com/), [Zoom](http://zoom.us/), [Webex](http://www.webex.com/), [talky](https://talky.io/), [sqwiggle](https://www.sqwiggle.com/), or [Google Hangouts](http://www.google.com/hangouts/). ## Screen Sharing This is your fast-track to remote pair programming, troubleshooting, hallway testing, and collaborating with other disciplines. The important features are the same as for audio/video calls: one-on-one sharing, group sharing, and the ability to do either in 60 seconds or less. Try [Screenhero](https://screenhero.com/) (for [Slack](https://slack.com/)), [HipChat](https://www.hipchat.com/), [join.me](https://join.me/), or [Webex](http://www.webex.com/). ## Development Tools ### Commit Messages > "The only difference between science and screwing around is writing it down." – [Adam Savage](https://twitter.com/donttrythis), Mythbusters Commits are like emails to future developers. Please write good commit messages for our _future_ selves. And not just ourselves, but any future developers working on this code. _Think of it as time travel_. When [Marty wrote Doc that letter](http://backtothefuture.wikia.com/wiki/Marty%27s_letter), it wasn’t a cryptic, short message he would only understand in-context. It wasn't simply a P.O. Box address for Doc to go find the details at. It was a clear description of the problem and solution. It was helpful as soon as it was read. _Commits are always in sync with the code_. This is awesome. It means we can write documentation about our code without the worry of it getting stale. That's better than comments, even. A commit message for a piece of code lasts exactly as long as the code it's talking about lasts. More on commit messages: - Alex 'Skud' Bayley, ["Start Your Commit Message with a Verb"](http://infotrope.net/2013/08/24/start-your-commit-message-with-a-verb/) - Stephen Ball, ["Deliberate Git"](http://rakeroutes.com/blog/deliberate-git/) ([video](https://vimeo.com/72762735)) ### Code Review Written, asynchronous code review is a central part of any solid development cycle. Don't take my word for it. From the excellent book, [Code Complete](http://www.amazon.com/Code-Complete-Practical-Handbook-Construction/dp/0735619670): > …software testing alone has limited effectiveness -- the average defect detection rate is only 25 percent for unit testing, 35 percent for function testing, and 45 percent for integration testing. In contrast, **the average effectiveness of design and code inspections are 55 and 60 percent.** Case studies of review results have been impressive: > > - In a software-maintenance organization, 55 percent of one-line maintenance changes were in error before code reviews were introduced. After reviews were introduced, only 2 percent of the changes were in error. When all changes were considered, 95 percent were correct the first time after reviews were introduced. Before reviews were introduced, under 20 percent were correct the first time. > - In a group of 11 programs developed by the same group of people, the first 5 were developed without reviews. The remaining 6 were developed with reviews. After all the programs were released to production, the first 5 had an average of 4.5 errors per 100 lines of code. The 6 that had been inspected had an average of only 0.82 errors per 100. Reviews cut the errors by over 80 percent. > - The Aetna Insurance Company found 82 percent of the errors in a program by using inspections and was able to decrease its development resources by 20 percent. > - IBM's 500,000 line Orbit project used 11 levels of inspections. It was delivered early and had only about 1 percent of the errors that would normally be expected. > - A study of an organization at AT&T with more than 200 people reported a 14 percent increase in productivity and a 90 percent decrease in defects after the organization introduced reviews. > - Jet Propulsion Laboratories estimates that it saves about $25,000 per inspection by finding and fixing defects at an early stage. Copied from [Coding Horror's copy](http://blog.codinghorror.com/code-reviews-just-do-it/). Yes, you still need to write tests. Automated testing is not enough to find defects early. Combining them with code review is a winning strategy. Code review reduces defects, increases learning and knowledge sharing, and serves as a written artifact of technical decisions made over time. Some options for deploying code reviews include [GitHub Pull Requests](https://help.github.com/articles/using-pull-requests/), [Gerrit](https://code.google.com/p/gerrit/), [ReviewBoard](https://www.reviewboard.org/), [Code Collaborator](http://smartbear.com/product/collaborator/overview/) More on code review: [Derek Prior](https://twitter.com/derekprior), ["Implementing a Strong Code-Review Culture" (video)](http://confreaks.tv/videos/railsconf2015-implementing-a-strong-code-review-culture). ### Digital Progress Board Sticky notes don't scale. I'll caution you here: this is not an opportunity to over-architect your development process. You need a few queues to understand the workload of each step in your development pipeline. This gives you insight into capacity and constraints, and provides a high-level picture of work in progress. Try [Trello](http://trello.com/), [Pivotal Tracker](http://www.pivotaltracker.com/), [JIRA's Agile Plugin](https://www.atlassian.com/software/jira/agile#!), [Waffle](https://waffle.io/), or [GitHub Issues](https://github.com/blog/831-issues-2-0-the-next-generation). ## Collaborative Writing Being able to take notes together in meetings is very powerful. A real-time document sharing tool brings people together in meetings and serves as a very powerful alternative to a whiteboard. Someone can decide to take live notes while others cleanup structure and edit behind them. Everyone can see the conversational history and it can be preserved for posterity. Preservation should take place elsewhere. These tools usually aren't good choices for long-term, curated document stores. Instead it may make sense to copy your artifacts from here to a story, or a `README`, or a specific document under a `docs/` directory in a repository. Try [Etherpad](http://etherpad.org/), [Hackpad](https://hackpad.com/), [Google Docs](https://docs.google.com/), [GitHub Wiki](https://help.github.com/articles/about-github-wikis/), or [Confluence](https://www.atlassian.com/software/confluence/#!). ## File Store Sharing large files is hard, still. Especially at work. It might be PSDs, PDFs, or OVAs. Making this easier will ease a common pain point of distributed teams. The solution you choose will depend on your particular risk profile. Try [Dropbox](http://dropbox.com/), [Box](https://www.box.com/), [NFS](), [WebDAV](http://webdav.org/), or [Google Drive](https://encrypted.google.com/drive/). ## Shared Calendar Availability must be constantly communicated. Picking a time to get people together across locations and timezones can be a challenge. Shared calendars solve that problem. Frankly, I wish there were more options here. If you know of any solid alternatives I'd love to hear about them. Try [Exchange](https://en.wikipedia.org/wiki/Microsoft_Exchange_Server) or [Google Calendar](https://encrypted.google.com/calendar). ## Email No, not really. Email is often one of the slowest, least effective means of collaboration and I don't recommend relying on it for that purpose. It does have a place, and I feel that place is in non-critical communication with no time pressure. For example, when you want to chat with someone who isn't "in the office" at that particular moment and it's not important enough to call them. It's also an effective way to receive reports and notifications which, often, will lead you to the other collaboration tools mentioned in this document. If you need a quick answer from a co-worker or team and you're starting to write an email I have this advice: **stop.** You're about to waste time and introduce needless delay. I bet you'll get a faster answer on text chat. # Techniques ## Everyone Should Experience Remote At some point you should send every team member home, or to their local coffee shop, or into separated conference rooms to simulate being remote. Empathy is a great motivator, and if the folks at headquarters are having trouble adjusting to your remote workforce this is a fast-track way to help. I love the idea of having everyone work from home (or wherever) every Friday. A policy like this tends to make people happy. They like it. However, it's not a big enough burden to change minds most of the time. It'll become your "heads down day" and the point of the exercise will be lost. I recommend at least a full week. Everyone is remote. Meetings continue as scheduled. So do pair programming and planning sessions. Standup? Yes. Meetings with other departments? Yes, plan to dial in or video conference. There's a higher-level business benefit to doing this regularly, too. It can serve as a good, real test of your operational business continuity plans. It's a sustained use of the tools you've put in place for when your office burns to the ground. You are thinking about how to operate the company when your office is on fire, right? There are team members for whom home is not a suitable work environment. Speaking as a dad who had toddlers I can tell you sometimes it just isn't going to be viable. Don't require everyone to be at home. For folks who can't they should be offered the option of renting a desk at a co-working space (to be expensed), or commandeering a conference room for the week. If they stay in the office, in a conference room, set a rule of no talking in person. Stay true to the simulation. ## Face Time (Onsite) Get people together. Relationships are best created in person and can be sustained remotely for a while. Minimum twice a year. Being together is too valuable to waste on business as usual. Close the laptops and get to know each other for a while. ## Invite Remotes to Hallway Conversation It is inevitable, and still good and healthy, that you'll strike up an ad hoc conversation at your desk. It'll probably be about work, and chances are someone in another location is interested in that topic. This is a moment to invite them in. How do you do that when they're in another timezone? A few ways. Invite them on video chat. This is the best. It's like you're right there in the room because your head is! Unplug your headphones so they can hear everyone and everyone can hear them. You can also invite them to audio, or you can all hop on chat and have the conversation in text. Video is still the highest bandwidth option and for that reason I recommend it. If you work in an open office you're probably saying something I hear all the time, _"but we'll need to find a conference room because it's rude to be on a call in the pit."_ No, it's not. It's no more rude than the conversation you're having right now. Will it surprise people? Yes, at first. They'll get over it. This is a stigma that should be eliminated. It's also what headphones were invented for. ## Over Communicate Your working relationships will benefit from the old adage "always over-communicate", just like any relationship. ### Share Your Personality (Off Topic) A benefit to being co-located with your peers is they get to know you. They learn about your kids, your hobbies, and your affinity for mango flavored tea. Even if you never said anything about it out loud they'd learn about you just from being nearby. Remote co-workers don't have the luxury of osmosis. Our relationships grow over time through familiarity and comfort. The more we know someone the closer we feel. That's why it's important to share your personality. Share it a lot. How? Off topic. Every organization needs an Off Topic chat room. This is where the stream of nonsense we all spew every day goes to live. This is where you tell someone you're getting coffee, where you post pictures from last weekend's mountain biking adventure, or where you drop a link to the latest badass thing [Neil deGrasse Tyson](https://twitter.com/neiltyson) just said. Off Topic is a safe place to chat about whatever you want with your co-workers. You may end up with more off topic channels, for specific interests, than work-specific channels. That's okay! I see channels for music, cooking, exercise, and DIY often. Let this thrive and grow organically. For those of you who lead remote or distributed organizations: (with the exception of offensive content) don't monitor or question the traffic in this room. There will probably be a lot of it, and if you've forgotten what it's like to build a thing you'll probably also wonder how these people get anything done all day. Don't worry about it. This is work. Instead of hearing it in the halls you're seeing it on the screen. No big deal. ### Have a Visible Pulse When you're remote it's your responsibility to exhibit a visible pulse. You may think nobody cares about every time you make a pot of coffee or push a branch, and you're probably right about that. What they do care about is knowing you're alive, there, and available when needed. Sending those seemingly useless messages is sending them a more important signal: you are reliable and okay. ## Pick a Timezone Your company probably has a home office. It can be helpful to agree that its timezone is your _Standard Operating Time (SOT)_. I've employed this technique before and it works. When is standup every day? _10am SOT_ _I'll be leaving the office a little early today so reach be before 1PM SOT if you need anything._ When discussing time you should always be explicit about timezones. The _Standard Operating Time_ technique provides an generalized solution. Each individual can translate SOT to their native timezone. After a little practice it'll be second nature. # Conway's Law Durable communication exhibits the same characteristics as accidental, convenient communication in a co-located space. The powerful difference is how inclusive, transparent, and reliable it is. Durable communication tools and techniques not only scale well with your organization, they'll empower your organization scale well, too. [Conway's law](https://en.wikipedia.org/wiki/Conway%27s_Law) states that _"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations"_. The health and quality of your product will be a direct reflection of the health and quality of your organization. Take great care of your organization and it will pay off in your product. [^yagni]: This is [YAGNI](https://en.wikipedia.org/wiki/YAGNI) in action, in case you were wondering. --- This is an excellent conference talk laying out the core problem with reliance on integrated tests[^integrated_tests]. I've seen this problem at several companies. Here are the steps to reproduce: 1. Build a complicated system organically. 2. Observe quality degrade. 3. Declare testing a panacea. 4. Build a large, QA-lead integrated test suite at great cost. Now a huge, brittle test suite exists which provides no direct value to development, where it's needed most, and doesn't address the root problem: poor system design. ## Integrated test hell. [J.B. Rainsberger](http://www.jbrains.ca/) describes the problem well. If your project has a monolithic, external test suite which relies on the entire architecture to run you are in this special hell right now. The clearest representation I can think of is his explanation of how many tests you'd have to write to get value out of an _integrated tests_ versus _isolated tests_[^isolated_tests]. A software architecture with a few interconnected components (lets say, for example: a database, REST API, UI, Job Queue) requires tests for each function of each component. If we're relying on _integrated tests_ we have to write tests to exercise every function of every component, and every combination of connections between every function of every component. As you write that software you need to write $$ O(n!) $$ integrated tests. You can't write that many tests. You don't have enough time to write enough tests to have confidence in that system. That's scary. **That's impossible.** ## The cycle. I love this because it's clear and true. We've all seen this run-on sentence loop: **100% of our integrated tests pass but there's a mistake[^mistake] in our software**; so we write more integrated tests to fill in the cracks which allows us to design more sloppily and gives us more opportunities for mistakes, and spending time on integrated tests means less time for isolated tests which increases the likelihood that **100% of the tests pass but we still have mistakes.** ## What should you do about this? There is a strong correlation between large numbers of integrated tests and design problems. Integrated tests don't offer any pressure to improve our designs. Isolated tests do. Stop pretending integrated tests are helping you. Write isolated tests. To quote J.B.: > _The real benefit of isolated tests — testing one function at a time — is that those tests put tremendous pressure on our designs. Those tests are the ones that make it most clear where our design problems are. **Remember that the whole point of test driven development is not to do testing; it's to learn about the quality of our design.** We use the theory that if our design has problems then the tests will be hard to write. The tests will be hard to understand. It'll be difficult to write these small, isolated tests to check one thing at a time._ If you have more _isolated tests_ than _integrated tests_ chances are you have a decent design with clear interfaces and contracts between collaborating systems. This path is cheaper, faster, less likely to allow mistakes, and provides high-bandwith feedback on the quality of your software design. As you write this software you need to write $$ O(n) $$ isolated tests. You don't have to multiply the code paths in your system to get thorough coverage, you can just add them. You go from a combinatorial explosion of tests-to-code-paths to a linear increase in tests. **That's possible.** There's a lot of gold in this talk. Watch it. Twice! [Watch on Vimeo](https://vimeo.com/80533536) [^integrated_tests]: Not to be confused with _integration tests_ (referred to in this talk as _collaboration tests_). Integrated tests require a complete, integrated architecture to run. _Integration tests_ simply test the collaboration between independent components. [^isolated_tests]: _Isolated tests_ is a good name for tests operating on a specific function within a software architecture which exercise that function directly, in isolation, independent of any external collaborators in the architecture. [^mistake]: Sometimes we call these defects or bugs; I agree with the speaker that's too abstract. It's a human error (more likely a series of human errors). Everywhere else in the world we call those mistakes. --- I prefer values which connect us deeply over rituals and traditions only some of us are able to enjoy. ## Culture fit is broken. The words we use to describe culture matter. We talk about the rituals and traditions we enjoy together. In agile software development one of the traditions we like is standups. Some of our rituals include drinking coffee and alcohol. It's very important to us. I'm a self-described "beverage enthusiast" so I'm no stranger to these rituals. These are ways to bring some people together, but when we're creatively building something we must dig deeper to describe what we value in our collaborative interactions. Here are some rituals and traditions that might bring us together. _"She doesn't like SVN either,"_ perhaps at an interview, convinces us she's like-minded. Do you enjoy ping pong? When I worked at [Pivotal](http://pivotal.io), we played a lot of ping pong. I'm not excited about it, I'm not very good at it, but ping pong doesn't make me good at my job. I enjoy drinking great beer socially! > Never in the history of my career has my ability to drink beer made me better at solving a business problem. I would venture a guess that's true for you, too. That's too superficial. Let's think more deeply about the values which really bind us together when we're creative and building things. ## What do you value? Here are some values which shape the my decisions about which organizations and communities I create with. I value _blame-free failure_. I value working with people who are willing to take risks, in an organization that's willing to take risks. If I can take big risks there might be big rewards, but there might also be big mistakes, and that's ok. I want to work with people who are comfortable with that. I value _growth through mistakes_ which are, often, our best learning opportunities. I love _sharing._ I like sharing success, responsibility, work, and time. Most of all I love sharing ideas, and working with people who are constantly putting their ideas out there. I mentioned taking risks and making mistakes. I value working with people tell me how they feel about e work I've done, let me know the good and the bad, so I can grow from that. I want to provide the same r others. I value _continuous learning_ through _continuous feedback._ I don't want problems to pile up each other and become big. I like _balance._ > My idea of work-life balance is not the same as yours and that's ok. I have children, not everyone has children, but I can't work 16 hours a day even though I love what I do. Maybe you can and that works for you but we should try to build a culture that can support both lifestyles, and more. These are some of things that really bind us together when we're working, they really matter. But when we're in an interview or considering what sort of community to build we tend to focus on whether or not we enjoyed coffee or had a good time at lunch. That's not really deep enough to build a strong community. ## Seek value fit. Consider how our values, the core principles of our character, fit together when building a community and working with one another. Not everyone likes coffee. Not everyone likes beer. Not every can pop off to the pub after work to make critical business decisions over a pint, so maybe we shouldn't be doing that. > This boils down to being deliberate about how we make decisions, and how we work together. Hegemony isn't an accident. If you aren't being deliberate about what values bring people in your community together then you're likely to end up with a community of people who are all the same. It's possible you will look different, or be different genders, but you're more likely to have the same experiences and lifestyles. This doesn't translate into a deeper level of inclusion. Consider the way our values bring us together rather than the rituals and traditions we occasionally enjoy. ## Seek inclusion. Rituals and traditions are helpful but superficial ways to bring people together. Unfortunately they only bring some people together. By all means have a good time at Whiskey Friday—I will—but don't fool yourself into thinking it's an inclusive event. In fact if everyone in your organization raises a glass at Whiskey Friday you've created a culture of rituals, not values, and that homogeneous culture is not diverse enough for me. ## Thank you. Thanks to the organizers of Ignite Velocity 2015 New York, the Pittsburgh Perl Workshop 2015, OSDC 2015, and Cloud Foundry Summit Berlin 2015 for giving me five minutes to talk about this at each of your events. I gave this talk four times in three weeks on three different continents. Recordings from [Velocity] and [OSDC] are available as are [the slides]. [Velocity]: https://www.youtube.com/watch?v=Kcmv-h2J3HQ [OSDC]: https://www.youtube.com/watch?v=BXv9KpYrW0w [the slides]: https://speakerdeck.com/caseywest/redefining-culture-fit --- Python development. Powerful, expressive, versatile... and sometimes, utterly chaotic when it comes to managing projects. If you've been developing in Python for any length of time, you've likely wrestled the multi-headed hydra: juggling different Python versions for different projects, untangling dependency conflicts, ensuring environments are reproducible, and just generally spending too much time fighting your tools instead of writing code. The Python ecosystem, in its vibrant, sprawling way, has offered many solutions over the years. We've had [pyenv](https://github.com/pyenv/pyenv) for managing Python versions, [venv](https://docs.python.org/3/library/venv.html) or `virtualenv` for environment isolation, [pip](https://pip.pypa.io/en/stable/) for package installation (often paired with `requirements.txt`), and more comprehensive tools like [Poetry](https://python-poetry.org/) and [PDM](https://pdm-project.org/en/latest/) aiming to manage the whole project lifecycle. Each solved parts of the puzzle, but often led to a fragmented toolbox – maybe you used `pyenv` + `Poetry`, or `venv` + `pip-tools`. It worked, but was it _fast_? Was it _simple_? For me the answer was increasingly "no." Setup felt complex, dependency resolution could be slow, and switching between projects required conscious effort to manage environments. I started searching for a better way, a more unified approach that could harness the speed of modern hardware and streamline the entire workflow. That search led me to **[uv](https://astral.sh/uv)**. This post is the story of that journey. We'll explore _why_ the Python environment hydra is so tricky, _why_ `uv` feels like the sharpest sword to tame it in 2025, and _how_ you can set up a development workflow inspired by my own, leveraging `uv` as a standalone tool for remarkable speed and simplicity. ## The Hydra's Heads: Why Is Python Management So Tricky? Before we crown a new champion, let's appreciate the challenges it needs to conquer: 1. **Multiple Python Versions:** Project A needs Python 3.9 for legacy reasons, Project B uses shiny new features in 3.12, and Project C needs testing across both. Installing Python globally is asking for trouble. Tools like `pyenv` solved this but required shell configuration and manual switching. 2. **Dependency Conflicts ("Dependency Hell"):** Project A needs `libraryX v1.0`, but Project B requires `libraryX v2.0`. Or worse, Project A depends on `libraryY` which needs `libraryX v1.0`, while Project B depends directly on `libraryX v2.0`. Managing these conflicts within isolated environments is key. 3. **Reproducibility:** How do you ensure your colleague (or your future self, or the production server) can recreate the _exact_ environment with the _exact_ versions of all dependencies (including dependencies _of_ dependencies)? Basic `pip freeze > requirements.txt` often isn't robust enough. Lockfiles are the answer, pioneered by tools like Poetry and pip-tools. 4. **Performance:** Waiting minutes for dependencies to resolve or install is a major productivity killer. Traditional tools, often written in Python themselves, can struggle with complex dependency graphs or large numbers of packages. This is where newer, performance-focused tools show their strength. 5. **Tooling Overload:** `pyenv` for versions, `venv` for environments, `pip` for installing, `pip-tools` for locking, `pipx` for installing CLI tools... The sheer number of tools needed for a "complete" setup adds cognitive overhead and setup friction. ## Enter uv: A Faster, Unified Challenger Developed by [Astral](https://astral.sh/) (the same folks behind the incredibly fast linter/formatter [Ruff](https://astral.sh/ruff)), `uv` is an extremely fast Python package installer and resolver, written in [Rust](https://www.rust-lang.org/). But its ambitions go far beyond just replacing `pip`. `uv` aims to be a **unified tool**, capable of replacing: - `pip` (package installation) - `pip-tools` (dependency locking via `uv lock`, `uv sync`) - `venv`/`virtualenv` (environment creation via `uv venv`) - `pyenv` (Python version management via `uv python install`, `uv python pin`) - `pipx` (installing and running CLI tools globally via `uv tool install`, `uvx`) Why is this compelling? - **Blazing Speed:** Seriously, it's fast. Dependency resolution and installation can be 10-100x faster than `pip` or Poetry. On an M-series Mac, this feels like unleashing the hardware's potential. - **Simplicity:** One tool to install, learn, and manage. Less configuration, fewer moving parts. - **Standards-Based:** It works seamlessly with [`pyproject.toml`](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/) (using the standard `[project]` table defined in [PEP 621](https://peps.python.org/pep-0621/)) and `requirements.txt` files. It generates universal lockfiles (`uv.lock`). - **Modern Features:** Built-in Python version management, dependency groups, environment management, script running (`uv run`), and global tool installation. While newer than some established tools, `uv` is maturing incredibly quickly, backed by significant resources and gaining massive community traction. As of April 2025, it feels like the most promising path towards a streamlined, high-performance Python development future. ## Crafting the Ideal Workflow: What Does "Good" Feel Like? My goal wasn't just to use a new tool, but to create a workflow that felt _effortless_. Based on my exploration and implementation (which uses [`zsh`](https://www.zsh.org/) and [Homebrew](https://brew.sh/) on macOS, managed with [chezmoi](https://www.chezmoi.io/)), the ideal workflow enabled by `uv` should provide: 1. **Automatic Python Version Switching:** `cd` into a project, and the correct Python version (project-specific or a global default) is instantly active _without_ needing `source .venv/bin/activate`. 2. **Easy Global Python Management:** A simple way to install Python versions and set a global default, perhaps even keeping that default automatically updated to the latest stable release. 3. **Seamless Project Management:** Creating new projects, adding dependencies, locking, and syncing environments should be quick, standard commands. 4. **Effortless CLI Tools:** Installing and updating global command-line tools written in Python (like `ruff`, `black`, `httpie`, or custom scripts) should be trivial. Let's build this. ## The Implementation Guide: Setting Up Your uv Environment These steps will guide you through setting up `uv` as your primary Python environment manager, inspired by my own setup but generalized for broader use. We'll primarily use `zsh` examples for shell integration, but provide pointers for others. **Prerequisites:** - A terminal (like macOS Terminal, iTerm2, WezTerm). - On macOS: [Xcode Command Line Tools](https://developer.apple.com/xcode/resources/). Install via `xcode-select --install`. (uv may need these to build Python or packages). - On Linux: Ensure you have necessary build tools (like `gcc`, `make`, `libssl-dev`, etc.). Consult your distribution's documentation or the [Python Developer's Guide](https://devguide.python.org/getting-started/setup-building/#build-dependencies). **Step 1: Install uv** The recommended way is the official script: ```bash curl -LsSf https://astral.sh/uv/install.sh | sh ``` Alternatively, use a package manager: - **macOS (Homebrew):** `brew install uv` - **Linux/Other:** Check the [official uv installation guide](https://astral.sh/uv/install.sh) for more methods (like `pipx`, `cargo`). After installation, **close and reopen your terminal** or source your shell profile (`source ~/.zshrc`, `source ~/.bashrc`, etc.). Verify the installation: ```bash uv --version # Should output the installed uv version ``` **Step 2: Install Python Versions** Use `uv` to install the Python versions you need. It automatically fetches native ARM64 builds on Apple Silicon. ```bash # Install specific versions uv python install 3.11 3.12 # List installed versions uv python list # See where uv installs Pythons (optional) uv python dir ``` You can optionally set a global default Python version that `uv` will use when no project-specific version is set: ```bash uv python pin --global 3.12 ``` **Step 3: Automate Global Python Updates (Optional, but Nice!)** Keeping your global default Python fresh requires occasional updates. We can automate checking for the latest stable release and updating if needed. Here's a script that finds the latest stable `cpython` version available via `uv`, compares it to your current global pin, and installs/pins the latest if it's newer: ```bash title="./update_global_python.sh" wrap {10,21,29} #!/bin/sh # Script to check and update the globally pinned uv Python version to latest stable echo "Checking global Python version..." # Find the latest stable CPython version (X.Y.Z format) available via uv # Filters for cpython-X.Y.Z--, extracts X.Y.Z, sorts, gets latest # Use --all-platforms to ensure visibility even if only different arch/os is installed LATEST_PYTHON=$(uv python list --all-platforms | grep '' | grep '^cpython-[0-9]\+\.[0-9]\+\.[0-9]\+-' | sed -n -E 's/^cpython-([0-9]+\.[0-9]+\.[0-9]+).*/\1/p' | sort -V | tail -n 1) if [ -z "$LATEST_PYTHON" ]; then echo "Error: Could not determine the latest stable Python version from 'uv python list'." >&2 # It's safer to exit non-zero if we can't determine the latest version exit 1 fi echo "Latest stable Python available: $LATEST_PYTHON" # Get the currently pinned global Python version (ignore errors if none is pinned) CURRENT_PYTHON=$(uv python pin --global 2>/dev/null || echo "") echo "Currently pinned global Python: ${CURRENT_PYTHON:-'None'}" # Compare and update if necessary if [ "$CURRENT_PYTHON" != "$LATEST_PYTHON" ]; then echo "Updating global Python from '${CURRENT_PYTHON:-'None'}' to '$LATEST_PYTHON'..." # Attempt to install and pin; exit non-zero on failure if uv python install "$LATEST_PYTHON" && uv python pin --global "$LATEST_PYTHON"; then echo "Successfully updated and pinned Python $LATEST_PYTHON." else echo "Error: Failed to install or pin Python $LATEST_PYTHON." >&2 exit 1 fi else echo "Global Python ($CURRENT_PYTHON) is already the latest stable version." fi exit 0 ``` **How to run this script?** - **Manually:** Save it as `update_global_python.sh`, make it executable (`chmod +x ./update_global_python.sh`), and run it periodically (`./update_global_python.sh`). - **Cron Job:** Schedule it to run daily/weekly using `crontab -e`. - **Automation Tools:** If you use tools like `chezmoi` (like I do), you can configure it to run automatically (e.g., using a `run_onchange_` script triggered by changes to the script itself, or a `run_always_` script). **Step 4: Integrate with Your Shell (Automatic Switching Magic!)** This is where we achieve the seamless `cd`-based environment switching, eliminating the need for `source .venv/bin/activate`. The core idea is to use a shell hook that runs every time you change directories (`chpwd` in zsh). This hook checks for a `uv`-managed Python (project-specific first, then global) and updates your `PATH` accordingly. **For zsh users:** 1. Add this function to your `~/.zshrc` (or a file sourced by it): ```zsh title="~/.zshrc" wrap {12,20,35,50} _update_uv_python_path() { # Check if uv command exists command -v uv &>/dev/null || return 0 local uv_python_path="" local uv_python_bin_dir="" local uv_root_dir="" local target_version="" # 1. Check for project-specific Python (.python-version file) # Redirect stderr to /dev/null to suppress "No project found" messages uv_python_path=$(uv python find . 2>/dev/null) # 2. If no project-specific Python, check for globally pinned Python if [[ -z "$uv_python_path" ]]; then # Get the globally pinned version string (e.g., "3.12") target_version=$(uv python pin --global 2>/dev/null) if [[ -n "$target_version" ]]; then # Find the actual path for that pinned version uv_python_path=$(uv python find "$target_version" 2>/dev/null) fi fi # 3. If still no uv-managed Python found for the context, potentially clear old paths and exit if [[ -z "$uv_python_path" ]]; then # Optional: Clean up any previous uv python paths if desired # uv_root_dir=$(uv python dir 2>/dev/null || echo "") # if [[ -n "$uv_root_dir" ]]; then # PATH=$(echo "$PATH" | awk -v RS=: -v ORS=: -v uv_root="$uv_root_dir" 'index($0, uv_root) != 1 {print}' | sed 's|:*$||') # fi return 0 fi # 4. Get the bin directory of the found Python uv_python_bin_dir=$(dirname "$uv_python_path") # 5. If the correct bin dir is already first in PATH, do nothing [[ "$PATH" == "$uv_python_bin_dir:"* ]] && return 0 # 6. Clean up old uv Python paths from PATH # Get the root directory where uv installs Pythons uv_root_dir=$(uv python dir 2>/dev/null || echo "") if [[ -n "$uv_root_dir" ]]; then # Escape potential special characters in path for awk/sed if necessary # Using a simple prefix check should be safe for typical paths. PATH=$(echo "$PATH" | awk -v RS=: -v ORS=: -v uv_root="$uv_root_dir" 'index($0, uv_root) != 1 {print}' | sed 's|:*$||') fi # 7. Prepend the correct Python bin directory to PATH export PATH="$uv_python_bin_dir:$PATH" } # Load the hook system and register the function to run on directory change autoload -Uz add-zsh-hook add-zsh-hook chpwd _update_uv_python_path # Run the function once immediately on shell startup _update_uv_python_path ``` 2. **Aliases (Optional but recommended):** Add these aliases to your `~/.zshrc` to ensure `python` and `pip` commands run within the context managed by `uv` when inside a project directory (it uses the virtual environment automatically): ```zsh title="~/.zshrc" wrap # Run python/pip via uv run if in a uv project context # Note: `uv run` automatically handles the virtualenv alias python='uv run python "$@"' alias python3='uv run python3 "$@"' # You might alias pip too, though using `uv add/remove/sync` is preferred # alias pip='uv run pip "$@"' ``` 3. **Restart your shell** or run `source ~/.zshrc`. **For Bash/Fish users:** - **Bash:** You'll need a different mechanism, often involving manipulating the `PROMPT_COMMAND` variable to run a function before the prompt is displayed. See the [Bash documentation](https://www.gnu.org/software/bash/manual/bash.html#index-PROMPT_005fCOMMAND) or community examples for `pyenv/nodenv`. - **Fish:** Uses functions triggered by changes to the `PWD` variable (e.g., `function --on-variable PWD my_hook`). See the [Fish documentation on event handlers](https://fishshell.com/docs/current/language.html#event-handlers). Consult the documentation for your specific shell. The [uv documentation](https://astral.sh/uv) or community discussions might also provide guidance or ready-made snippets. **Step 5: Managing Your Projects** Now, the day-to-day workflow becomes much smoother: 1. **Create a new project:** ```bash mkdir my_new_project cd my_new_project ``` 2. **Pin a Python version for this project (Optional):** If you don't want the global default. This creates a `.python-version` file. ```bash uv python pin 3.11 # Now, the shell hook (if active) will automatically use 3.11 here ``` 3. **Initialize the project:** Creates `pyproject.toml` and potentially `.venv`. ```bash uv init # Follow prompts for name, version, etc. ``` 4. **Add dependencies:** Updates `pyproject.toml` and installs into the implicitly managed `.venv`. ```bash # Add main dependencies uv add requests "flask>=2.0" # Add development dependencies uv add --dev pytest ruff black ``` 5. **Lock dependencies:** Creates `uv.lock` with exact versions of everything. Commit `pyproject.toml` and `uv.lock` to Git. ```bash uv lock ``` 6. **Sync environment:** Installs exact versions from `uv.lock`. Use this after cloning or pulling changes. ```bash uv sync # Use `uv sync --strict` (or similar flag, check `uv sync --help`) # to ensure the environment exactly matches the lockfile (removes extra packages) ``` 7. **Run code/commands:** Executes within the project's virtual environment _without_ manual activation, thanks to the shell hook or the `uv run` command (or our alias). ```bash # If using the alias: python src/my_app/main.py pytest # Explicitly using uv run (always works, even without the hook/alias): uv run python src/my_app/main.py uv run pytest uv run flask --app src/my_app:app run --debug ``` 8. **Update dependencies:** ```bash # Update specific package(s) in the lockfile to latest compatible uv lock --upgrade requests flask # Update all packages in the lockfile uv lock --upgrade # After updating the lockfile, sync the environment uv sync ``` **Step 6: Managing Global CLI Tools** `uv` can replace `pipx` for installing Python-based command-line tools globally. 1. **Install a tool:** ```bash uv tool install ruff uv tool install black uv tool install httpie ``` 2. **Run an installed tool:** Just type its name. ```bash ruff check . black . http --help ``` 3. **List installed tools:** ```bash uv tool list ``` 4. **Uninstall a tool:** ```bash uv tool uninstall ruff ``` 5. **Update tools:** ```bash # Update a specific tool uv tool install --upgrade ruff # Update all tools (requires scripting, see below) ``` 6. **PATH Configuration:** Tools are installed in a central location (`~/.local/bin` is common, check `uv tool dir --show-bin`). `uv`'s initial install script (`curl ... | sh`) usually handles this. If commands aren't found after installing `uv` or tools, ensure the `uv` bin directory (often `~/.local/bin`) is in your `PATH`. Your shell profile (`~/.zshenv`, `~/.zshrc`, `~/.bash_profile`, `~/.profile`, `~/.config/fish/config.fish`) might need a line like: ```bash export PATH="$HOME/.local/bin:$PATH" # Adjust path if needed ``` Then restart your shell. 7. **Automate Tool Updates (Optional):** You can create a simple script to keep a list of desired global tools up-to-date. ```bash title="update_global_tools.sh" wrap {25} #!/bin/sh # Script to install/update a list of global Python tools using uv # Define tools in a multi-line string, one tool per line TOOLS=" ruff black httpie gitingest # Add other desired tools here, ensuring no trailing spaces " echo "Updating global Python tools via uv..." # Use `echo` and `while read` to iterate over lines safely echo "$TOOLS" | while IFS= read -r tool || [ -n "$tool" ]; do # Skip empty lines or lines starting with # case "$tool" in ''|\#*) continue ;; esac echo "Ensuring $tool is installed and up-to-date..." # Use -U as a shorthand for --upgrade # Add error checking if uv tool install -U "$tool"; then : # Success, do nothing else echo "Warning: Failed to install/update $tool." >&2 # Decide if you want to exit or continue # exit 1 # Exit script on first failure # Or just continue with the next tool (current behavior) fi done echo "Global tool update process finished." exit 0 ``` Save this (e.g., `update_global_tools.sh`), make it executable (`chmod +x ./update_global_tools.sh`), and run it manually (`./update_global_tools.sh`), via cron, or through automation like `chezmoi`. ## Switching Tracks: Migrating Your Existing Projects to uv Adopting `uv` for new projects is straightforward, but what about your existing codebase? Migrating from established setups like `requirements.txt` or Poetry is definitely feasible, and `uv` provides commands to help smooth the transition. Let's look at the common scenarios. **Scenario 1: Migrating from `requirements.txt` (often with `venv` + `pip`/`pip-tools`)** This is perhaps the most common setup for many existing applications. You likely have a `requirements.txt` file (maybe generated by `pip freeze` or `pip-compile`) and potentially separate files like `requirements-dev.txt`. 1. **Navigate to your project directory:** ```bash cd path/to/your/existing_project ``` 2. **Pin the Python Version (Recommended):** Ensure `uv` knows which Python version this project needs. If you don't have a `.python-version` file yet, create one: ```bash # Replace with the actual version, e.g., 3.10 uv python pin ``` Make sure you have this version installed via `uv python install ` if needed. 3. **Initialize `uv`:** This creates the `pyproject.toml` file if it doesn't exist. `uv` will recognize existing virtual environments (like `.venv`) or create one. ```bash uv init # Answer the prompts for project name, etc., or edit pyproject.toml later ``` 4. **Add Dependencies from Requirements Files:** Use `uv add -r` to import dependencies directly from your existing files into `pyproject.toml`. ```bash # Add main dependencies uv add -r requirements.txt # Add development dependencies (if you have a separate file) # Make sure the --dev group matches your needs or adjust as necessary uv add -r requirements-dev.txt --dev ``` Review your `pyproject.toml` to ensure the dependencies under `[project.dependencies]` and `[tool.uv.dev-dependencies]` (or other groups you might define) look correct. 5. **Generate the `uv` Lockfile:** Create the definitive `uv.lock` based on the dependencies now listed in `pyproject.toml`. ```bash uv lock ``` 6. **Sync Your Environment:** Install everything specified in the new `uv.lock` into your virtual environment (`.venv`). ```bash uv sync # Use --strict if desired to remove packages not in the lockfile ``` 7. **Cleanup:** Once you've verified everything works (run your tests!), you can safely: - Delete the old `requirements.txt` and `requirements-dev.txt` files. - Commit the new `pyproject.toml` and `uv.lock` to version control. - **Crucially:** Update any CI/CD pipelines, Dockerfiles, or deployment scripts. Replace commands like `pip install -r requirements.txt` with `uv sync`. **Scenario 2: Migrating from Poetry** Poetry also uses `pyproject.toml` but has its own `[tool.poetry]` section and `poetry.lock` file. `uv` uses the standard `[project]` section and its own lockfile format. 1. **Navigate to your project directory:** ```bash cd path/to/your/poetry_project ``` 2. **Pin the Python Version:** Just like before, ensure `uv` knows the target Python version: ```bash uv python pin # e.g., 3.11 ``` 3. **Convert `pyproject.toml`:** This is the most manual step. `uv` doesn't automatically convert Poetry's specific format. You need to translate the metadata and dependencies from the `[tool.poetry]` section to the standard `[project]` and `[tool.uv.dev-dependencies]` sections. - **Project Metadata:** Copy fields like `name`, `version`, `description`, `authors`, `license`, `readme` from `[tool.poetry]` into the corresponding fields under `[project]` ([PEP 621 standard](https://peps.python.org/pep-0621/)). - **Main Dependencies:** Move dependencies listed under `[tool.poetry.dependencies]` to `[project.dependencies]`. - **Development Dependencies:** Move dependencies from `[tool.poetry.group.dev.dependencies]` (or similar Poetry groups) to `[tool.uv.dev-dependencies]`. If you have other groups, map them similarly under `[tool.uv.tool..dependencies]`. - **Build System:** Check the `[build-system]` table. Poetry uses `poetry-core`. You can keep this if you still want to use Poetry's build capabilities _alongside_ `uv` for environment management, or switch to another backend like `hatchling` or `setuptools` if you plan to use `uv build` (which invokes the specified backend). - **Remove Poetry Section:** Once everything is migrated, delete the entire `[tool.poetry]` section from `pyproject.toml`. **Example `pyproject.toml` Conversion:** Let's say your original `pyproject.toml` looked something like this (simplified): ```toml title="pyproject.toml" # Original Poetry pyproject.toml (Before Migration) [tool.poetry] name = "my-awesome-app" version = "1.2.3" description = "Does awesome things." authors = ["Dev Team "] license = "Apache-2.0" readme = "README.md" [tool.poetry.dependencies] python = "^3.10" flask = "^2.1" requests = ">=2.25,<3.0" [tool.poetry.group.dev.dependencies] pytest = "^7.0" black = {version = "^23.0", optional = true} # Example optional within group [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api" ``` After manually converting it for `uv`, it would look like this: ```toml title="pyproject.toml" # Converted uv-compatible pyproject.toml (After Migration) [project] name = "my-awesome-app" version = "1.2.3" description = "Does awesome things." authors = [ { name = "Dev Team", email = "dev@example.com" }, ] license = { text = "Apache-2.0" } # Or reference file: { file = "LICENSE" } readme = "README.md" requires-python = ">=3.10" # Translate Python constraint # Main dependencies moved here dependencies = [ "flask>=2.1,<3.0", # Poetry's ^2.1 becomes >=2.1,<3.0 "requests>=2.25,<3.0", # This constraint translates directly ] # Dev dependencies moved here (standard location) [project.optional-dependencies] dev = [ "pytest>=7.0,<8.0", # Poetry's ^7.0 becomes >=7.0,<8.0 "black>=23.0,<24.0", ] # Alternatively, uv also supports: # [tool.uv.dev-dependencies] # pytest = ">=7.0,<8.0" # black = ">=23.0,<24.0" # Build system - kept Poetry's, or could switch [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api" # --- OR using e.g. Hatchling --- # requires = ["hatchling"] # build-backend = "hatchling.build" # Notice the [tool.poetry] section is completely gone. ``` Pay close attention to translating version specifiers (like Poetry's `^` or `~`) into the standard specifiers (`>=`, `<`, `==`, etc.) expected in `[project.dependencies]`. You might need to consult the [PEP 440 specification](https://peps.python.org/pep-0440/#version-specifiers) for details. Using the standard `[project.optional-dependencies]` for development dependencies is generally recommended for better compatibility with other tools, although `uv` also directly supports `[tool.uv.dev-dependencies]`. _Tip:_ You could run `uv init` in a temporary directory to see the structure `uv` expects in `pyproject.toml` and use that as a template. 4. **Generate the `uv` Lockfile:** This is crucial. `uv` will read your _newly formatted_ `pyproject.toml` and generate a `uv.lock` file from scratch. **It completely ignores the old `poetry.lock` file.** ```bash uv lock ``` 5. **Sync Your Environment:** Install dependencies based on the new `uv.lock`. ```bash uv sync ``` 6. **Cleanup:** After testing and confirming the migration: - Delete the old `poetry.lock` file. - Commit the modified `pyproject.toml` and the new `uv.lock`. - Update CI/CD and other scripts: Replace `poetry install` with `uv sync`, `poetry run ` with `uv run `, `poetry build` with `uv build` (if using a compatible backend), and `poetry publish` with `uv publish`. Migrating takes a bit of focused effort, especially converting `pyproject.toml` from Poetry. However, the payoff is consolidating your workflow around `uv`'s speed and simplicity for dependency and environment management moving forward. Remember to test thoroughly after migration! ## Conclusion: A Simpler, Faster Python Future Wrestling the Python environment hydra has felt like a rite of passage for too long. While tools like `pyenv` and `Poetry` were valiant efforts, the rise of `uv` feels like a paradigm shift. By unifying version management, environment handling, dependency resolution, locking, and global tool installation into a single, lightning-fast binary, `uv` drastically simplifies the Python developer experience. Adopting the standalone `uv` workflow, especially combined with shell integration for automatic environment switching, has significantly boosted my productivity and reduced daily friction. Projects initialize faster, dependencies install in seconds, and managing Python versions or CLI tools becomes trivial. Is there a learning curve? A little, especially when migrating existing projects or setting up the shell integration. But the payoff – a development environment that feels fast, cohesive, and almost invisible – is well worth the initial effort. Give `uv` a try. Tame the hydra. Spend less time fighting your tools and more time building amazing things with Python. The future of Python development is looking remarkably fast and refreshingly simple. --- ## References and Further Reading - **uv:** [Homepage](https://astral.sh/uv) | [GitHub Repository](https://github.com/astral-sh/uv) | [Installation Guide](https://astral.sh/uv/install.sh) - **Ruff (Companion Linter/Formatter):** [Homepage](https://astral.sh/ruff) - **Python Packaging Standards:** - `pyproject.toml`: [pip docs](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/) - PEP 621 (Project Metadata): [python.org](https://peps.python.org/pep-0621/) - PEP 517 (Build Backends): [python.org](https://peps.python.org/pep-0517/) - PEP 518 (Build System Requirements): [python.org](https://peps.python.org/pep-0518/) - **Traditional Tools (Mentioned for Context):** - [pyenv](https://github.com/pyenv/pyenv) - [venv](https://docs.python.org/3/library/venv.html) - [pip](https://pip.pypa.io/en/stable/) - [Poetry](https://python-poetry.org/) - [PDM](https://pdm-project.org/en/latest/) - [pip-tools](https://github.com/jazzband/pip-tools) - [pipx](https://pipx.pypa.io/stable/) - **Shells:** [zsh](https://www.zsh.org/) | [Bash](https://www.gnu.org/software/bash/) | [Fish](https://fishshell.com/) - **Other Tools Mentioned:** [Homebrew](https://brew.sh/) | [chezmoi](https://www.chezmoi.io/) | [Rust](https://www.rust-lang.org/) | [Xcode Command Line Tools](https://developer.apple.com/xcode/resources/) --- _Disclaimer: The Python tooling landscape evolves rapidly. This post reflects the state and my recommendation as of April 2025. Always consult the official documentation for the latest features and commands._ --- Asking for help can be hard for some of us. For others it may come too quickly. Wait an hour[^anhour] to ask for help. No more, no less. This is advice I give to my team members regardless of experience, background, or confidence level. There are two primary behaviors when asking for help. Waiting too long or not waiting long enough. These behaviors are opposite but the advice is the same: one hour. This is how it usually sounds. | Head Banging | Hand Throwing | | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | I know you want to figure this out on your own. Given enough time you surely can. Try this: | I know you're confused. This is a new problem! You've never done this before! Try this: | | | | | _When you find yourself beating your head on the wall set a timer for an hour. When time is up ask for help immediately. We're here for you and you'll get that help and move on to the next challenge._ | _When you find yourself giving up and throwing your hands in the air set a timer for an hour. When time is up ask for help immediately. We're here for you and you'll get that help and move on to the next challenge._ | It doesn't matter which stereotypical behavior you identify with. Whether you're giving in after an hour or forcing yourself to wait an hour it's time well spent. ### The Benefit of Waiting One Hour You will read so much about the technologies, libraries, and protocols giving you trouble. Maybe you won't find the answer but you will learn a ton anyway and that knowledge has value. Struggling for an hour, when done properly by researching and trying different approaches, is a valuable learning experience. ### The Benefit of Asking for Help There's an inflection point when struggle becomes spinning your wheels and wasted effort. There's no honor lost in stopping and asking for help. At some point it's time to bring other brains into this. Your team is your support system. Use them! After an hour of dedicated effort, it's often more productive to ask for help. ### Making it Safe _The Help Timer_ works best in a healthy environment. There's a give and take to it. If you wish a teammate would ask for help sooner, or another teammate would spend a little more time on the problem before asking for help, you have to do your part to make that a safe thing to do. Practicing this approach to asking for help requires trust in your teammates. It's the foundation of any healthy team.[^fivedysfunctions] Trust is built through practice. It's built through the continual reception of positive responses to acts of vulnerability. When your teammate exhibits vulnerability you must respond positively. ### An Act of Vulnerability If this resonates with you it might feel uncomfortable. I get it. Asking for help is a vulnerable act. So is struggling! Being vulnerable can be hard, and it can feel alien, but it's worth it! I encourage you to watch the following [excellent TED talk by Brené Brown](http://www.ted.com/talks/brene_brown_on_vulnerability) ([full transcript](http://www.ted.com/talks/brene_brown_on_vulnerability/transcript?language=en)). She describes the benefits of embracing vulnerability. > "There was only one variable that separated the people who have a strong sense of love and belonging and the people who really struggle for it. And that was, the people who have a strong sense of love and belonging believe they're worthy of love and belonging. That's it. They believe they're worthy." | Head Banging | Hand Throwing | | :------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------- | | If you tried hard and still can't figure it out _it's okay and **you are worthy** of help._ | If you're lost even after looking at a bunch of maps _it's okay and **you are worthy** of help._ | [^anhour]: It doesn't _have_ to be an hour. 30 minutes, a whole day… you pick. [^fivedysfunctions]: I strongly recommend reading [The Five Dysfunctions of a Team by Patrick Lencioni](https://en.wikipedia.org/wiki/The_Five_Dysfunctions_of_a_Team). When there's an absence of trust we aren't willing to be vulnerable within a group. Great culture builds on trust and vulnerability. --- > Originally published by me on [December 18, 2006](https://web.archive.org/web/20070729012936/http://blog.caseywest.com/2006/12/theres_still_no_silver_bullet_1.html) and reprinted here as-is. I recently read [Frederick P. Brooks, Jr.'s](http://en.wikipedia.org/wiki/Fred_Brooks) essay [No Silver Bullet: Essence and Accidents of Software Engineering](http://faculty.salisbury.edu/~xswang/Research/Papers/SERelated/no-silver-bullet.pdf). Brooks wrote this paper 20 years ago and it still rings true today. I like the whole essay and encourage you to read it if you haven't already. Brooks is an [oft quoted](http://www.brainyquote.com/quotes/authors/f/frederick_p_brooks_jr.html) writer. I'm going to indulge in the practice by pointing out a few things from this essay that I think will always be true. > Likewise, a scaling-up of a software entity is not merely a repetition of the same elements in larger sizes, it is necessarily an increase in the number of different elements. In most cases, the elements interact with each other in some nonlinear fashion, and the complexity of the whole increases much more than linearly. > > The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence. I really like this, because today there are [a lot](http://www.rubyonrails.org/) of [arguably excellent](http://www.djangoproject.com/) [attempts](http://www.turbogears.org/) to [abstract complexity](http://jifty.org/) in [modern application development](http://www.ning.com/). Many software developers look to frameworks and application environments to manage the complexity of software development for them. These developers expect incredible gains in productivity by using a tool that makes building applications "really easy." Don't be fooled. Tools help you build things, yes, but don't expect to get a pony when you install Ruby on Rails (although, as it happens, you do get a pony when you install Jifty). Frameworks are not 100% tools, they're 80% tools. If they were 100% tools then you wouldn't be making applications you'd be writing configuration files. What kind of programmer writes configuration files for fun? > In many cases, the software must conform because it is the most recent arrival on the scene. In others, it must conform because it is perceived as the most conformable. But in all cases, much complexity comes from conformation to other interfaces; this complexity cannot be simplified out by any redesign of the software alone. This most obvious form of complexity through conformity in web applications is browser compatibility. I don't think that is the scenario that takes the cake, however. Anything relating to or interfacing with email clients increases complexity in an extremely non-linear fashion. There are other issues, too, such as representing information for RSS and Atom feeds. A lot of cruft exists around the edges of software because it must conform to its environment. You cannot simplify conformity within your software alone. > In spite of progress in restricting and simplifying the structures of software, they remain inherently unvisualizable, and thus do not permit the mind to use some of its most powerful conceptual tools. This lack not only impedes the process of design within one mind, it severely hinders communication among minds. If you've ever worked with someone else when writing software you know this to be true. The inability to visualize software is a serious friction point between team members. When asked ["What do you think makes some programmers 10 or 100 times more productive than othes?"](https://web.archive.org/web/20070712044136/http://www.stifflog.com//2006//10//16//stiff-asks-great-programmers-answer//) [Peter Norvig](http://norvig.com/) answers, "The ability to fit the whole problem into their heads at one time." That's often a critical ability and it speaks to Brooks's admission that software design is hard enough within a single mind. [David Heinemeier Hansson](http://www.loudthinking.com/) has a complimentary answer to the same question, "The ability to restate hard problems as easy ones." Having the ability to clearly communicate software design is important, yes. I would add to these answers, "The ability to explicitly articulate the design of complex software." Having these attributes brings you closer to making the invisible visible. Imagine how important these skills must be if you're building a geographically distributed team like [37signals](http://37signals.com) or [Socialtext](http://socialtext.com)? The lowest common denominator for being hired on these teams is likely expressed in the previous paragraph. > I do not believe we will find productivity magic here. Program verification is a very powerful concept, and it will be very important for such things as secure operating-system kernels. The technology does not promise, however, to save labor. Verifications are so much work that only a few substantial programs have ever been verified. 20 years after writing this Brooks is still right. In context, Brooks is talking about "test first" methodologies here, asserting that testing during design and specification will not save you labor. He goes on: > More seriously, even perfect program verification can only establish that a program meets its specification. The hardest part of the software task is arriving at a complete and consistent specification, and much of the essence of building a program is in fact the debugging of the specification. In my opinion this statement is a key gem. I interpret this statement to be a nod to iterative development, suggesting that the imperfect art of testing our assumptions over time is a large part of the essence of software development. One could bastardize this as "release early, release often." I feel that's an oversimplification that leaves large room for error. Taking time to design software is critical. I agree with [Joel](http://www.joelonsoftware.com/articles/fog0000000036.html) when he says "Programmers and software engineers who dive into code without writing a spec tend to think they're cool gunslingers, shooting from the hip. They're not. They are terribly unproductive. They write bad code and produce shoddy software, and they threaten their projects by taking giant risks which are completely uncalled for." Let's skip to the end, to the real message within the message of the essence of software development: > Hence, although I strongly support the technology-transfer and curriculum development efforts now under way, I think the most important single effort we can mount is to develop ways to grow great designers. > > No software organization can ignore this challenge. Good managers, scarce though they be, are no scarcer than good designers. Great designers and great managers are both very rare. Most organizations spend considerable effort in finding and cultivating the management prospects; I know of none that spends equal effort in finding and developing the great designers upon whom the technical excellence of the products will ultimately depend. This, I believe, is the most important advancement since Brooks wrote his paper. Organizations have recognized the need to grow great software designers. As a result the pace of development and product creation has increased dramatically. The state of the world is much better now than 1986 but we still have a long way to go. --- I was talking with someone the other day about my time as (Interim) VP of Engineering at [Socialtext](http://socialtext.com). Did I enjoy that? The question was framed like this: some people just like doing things and not dealing with the social aspects of management. But I wonder, are they really much different? > Originally published by me on [March 25, 2009](https://web.archive.org/web/20090416185738/http://caseywest.com/2009/03/25/your-software-is-made-of-people/) and reprinted here as-is. Software development is creating, maintaining, and evolving a system. Use whatever action verb you like, you are working with a system. That system can be made better or worse by your actions. If you fix a bug the system is better. Remove a networking bottleneck? Better. Introduce a needless database query on every iteration of a loop? Worse. Software doesn’t work in isolation. The system is bigger than that. If you increase the memory requirements for your software the servers had better have enough memory to manage it. If you rewrite your code in Python a host of changes are required to make that change possible. How are teams much different? Leading a team requires the creation, maintenance, and evolution of a system. Again, you can make it better or worse. Help a peer solve a problem with a better tool then your system is better. Reduce needless process? Better. Introduce a needless process on every iteration of development? Worse. I think both people and technology are irrevocably intertwined. In fact, hacking on one and not the other will cause the performance of both to suffer. This is called [Sociotechnical Systems Theory](http://en.wikipedia.org/wiki/Sociotechnical_systems_theory). # Joint Optimization A team survives - and eventually thrives - through the _joint optimization_ of their sociological and technological systems. Improving one alone often leads to recessive tendencies in the other. The nature of a team is the symbiotic relationship between its people and technology systems. Success can’t be realized by improving technology alone. This concept is often hard for everyone. Technologists find it easy to ignore social aspects of an organization. Non-technical specialists are reluctant to consider the artificial reality of technical objects like software. So it can be hard to consider both technical and social aspects of a system. The delivery of meaningful value to customers requires the actions of both people and technical objects. One can’t improve without the other. [Technical achievement is equally as important as social advancement.](http://scholar.lib.vt.edu/ejournals/SPT/v4_n3html/ROPOHL.html) ## People are (part of) Technical Strategy [Hacking on the social realities in your technology team has strategic value.](http://alexandria.tue.nl/extra2/200211694.pdf) A healthy team can do more than generate fantastic technological innovations because a healthy team can more accurately assess the environment they’re in. A viable business strategy can’t simply focus on organizational capabilities as most technologists are prone to do. The environment your team operates in isn’t the primary strategic factor as many non-technical specialists see it. The decision isn’t either/or among organizational capability and environmental reality. The winning strategy is both/and: react to environmental realities within the context of current and improved organizational capabilities. ## The API is Different The major difference between people and software on a technical team is the API. You’re still debugging, refactoring, creating, evolving, and removing what you don’t need. As a technical team leader you need to talk to both types of interfaces. The API is very different for debugging people vs. debugging software. If you want to build world class software you have to build a world class team. This is also why it’s hard for a star programmer to become a star manager. They never spent time learning the People API. ## Footnote Some of this thinking was done as research for a previous company. I was asked in appropriately vague terms how to fix our software delivery process. The pain was that it took months to get even the smallest changes to the customer. When I searched for the root of the problem it became clear there were two intertwined problems: one technical and the other social. Half the company was looking for a quick technical fix that would make it all better. The other half wanted to add process to over come the social issues. It was obvious to me we would have to fix both if we really wanted to solve the problem. Any solution that ignored the fact that we were a socio-technical organization was lacking. ---