formae vs. Crossplane

declare resources as Kubernetes objects and the control plane creates them

built for Day 1 and Day 2+ operations

On Day 1, everything is simple, but a properly structured approach with IaC is needed - ClickOps doesn't work on Day 1 at scale, for example. Repeatability of initial rollout into different environments always plays an important role.

the control plane keeps reconciling for the lifetime of the resource

built for Day 1 and Day 2+ operations

Starting on Day 2, the reality is complicated, and the so-called "drift" is inevitable. Small changes are common, and also different tools will almost certainly be in use to manage infrastructure. Even ClickOps is a legitimate tool in some cases - there is no reason to avoid powerful tools such as Cloud Web Consoles just to satisfy a rigid process that is implemented for Day 1 operations.

Cloud providers make updates without updating or even knowing about your infrastructure code. All sorts of autoscaling rules and technologies constantly change the topology. Security tools and teams make urgent and planned changes often bypassing infrastructure code as well. Different teams follow different processes and implement different workflows for maximum speed. It is necessary to stay in control of drift and automate your response to it, not only be able to detect it.

• every user needs Kubernetes fluency before anything else

• the abstraction developers self-serve from has to be built by the platform team first

supports all use cases and workflows of operations and platform teams

Empowering every engineer to work with the tool enables faster change at lower cost. Infrastructure experts can focus on low-level detail such as networks, while less experienced engineers can still make targeted/small changes without going too deep. Developers - who barely operate systems - can consume or request infrastructure without knowing any low-level detail. All these roles have their own nuances, too, depending on the team setup, skills and situations. For example, being an expert and having to fix an urgent problem at 2a.m. is a different use case than preparing a new team environment, and requires different focus, blast radius control and sometimes even tools.

• everything is YAML, including the compositions that generate more of it

• patching YAML with more YAML was deprecated in v1.17, which is the project agreeing it did not work

• what replaced it embeds KCL or Python inside a YAML string, so nothing checks the seam between the two until the control plane runs it

• uses Pkl, which is a simple configuration language with minimalistic, but comfortable elements of programming

• open to further schemas

The language of the infrastructure code needs to be simple. Of course, "simple" is highly subjective. But for example, Ops teams usually don’t do full-blown software development, which most ecosystems such as TypeScript, Java or Python require. Scripting is as far as people go in Ops. So far, YAML is the widely used common denominator that every engineer accepts, but it requires extra tools for templating, conditionals, functions and other means borrowed from programming. And these are needed for flexibility and DRY (don't repeat yourself).

• resources are custom resource definitions with an OpenAPI schema, and the API server validates against it

• the compositions that generate them are not: the KCL or Python inside a YAML string is checked by nothing until the control plane runs it

• so half the configuration is schema-safe and half is a block of text

strong, uniform schema

Schema safety is essential to avoid human error. Understanding the schema without having to navigate through each provider's source code makes configuration of resources simple and allows team-specific derivatives. On the other hand, using such low-level languages such as YAML currently comes at the cost of not having any schema safety at all, with a high risk of human error - sometimes catastrophic.

• resources are schema-defined

• composing them is a function pipeline you write, run and maintain yourself

• simple resource declaration through classes and inheritance

• hints on resource types that are used to automatically identify and resolve dependencies

• provides a so-called res, which is a simple yet powerful referencing construct

When all data-specific, mechanical aspects, as well as dependencies of the resource are defined through schema, target infrastructure is easy to maintain and adjust, and it allows you to keep the infrastructure code clear and the providers / plugins lean and simple to maintain. Too often, relationship and dependency logic is hiding inside already complicated provider code, which unnecessarily complicates the resource lifecycle management and the usage of the tool.

• providers are controllers packaged for the cluster

• many are generated from cloud APIs and inherit the whole surface, warts included

• plugins are simple and uniform

• every technology plugin implements CRUD+

• workflow, state transitions, retries etc. are implemented through the agent, so the plugin just works with data without managing state

Low provider (or plugin) complexity means easy update cycle, reliability and ability for anyone in the community to quickly implement their own plugins and provide bugfixes to existing ones. In the classic IaC, providers are often full-blown programs with workflow and lifecycle logic which can usually only be adjusted by its creators. There are also big differences in provider quality that are difficult to deal with. Lightweight plugins also relax that issue.

controllers run in the cluster and keep working after you close the laptop

• active agent

• takes care of state management and automation

• picks up unfinished work after failures

Engineers shouldn’t need extra tools to compensate for the inability of their IaC tool to support automation. Classic IaC tools attempt to work exclusively on the local machine, following the minimalistic philosophy of the original UNIX tools. This hinders the ability to work in teams or multiple teams, and puts the user in the middle of the process as yet another tool. True automation is only possible by handing off work to an active agent, removing toil for users.

reconciliation is asynchronous by construction

• executes every stack and resource change asynchronously

• the agent makes sure every resource change is going through a set of finite state machines that eventually finish

Applying infrastructure changes asynchronously makes them reliable and repeatable. Some infrastructure modifications in some technologies can take a long period of time and follow a complicated workflow. This doesn't mean that a full-blown workflow engine is the solution though - it only means that infrastructure changes need to run asynchronously, without the initiating user process having to wait for the result. And that unfinished work can be picked up after crashes without the risk of state corruption.

• state lives in the cluster and is managed for you

• the cluster is yours to run, back up, secure and upgrade

• automatically manages state

• stored in an RDBMS for strong durability and consistency

State management is a highly manual process in the classic IaC, and constantly requires human attention and intervention. Even if some tools manage the state data in cloud offerings, the point in time when the state is updated is still up to the engineer, making them part of the tool. Engineers should not have to tell their IaC tool when to persist or update its state. Having to manually maintain state introduces an additional layer of complexity and possible human and technical error.

reconciliation happens per resource

locks per resource

Global locking is how classic IaC tools make sure their state doesn't get corrupted by parallel mutations, but only a single resource can be locked in the majority of cases. Global locking is causing slowdowns and waterfall-like team workflows, while minimal locking makes sure teams and engineers can move independently and fast.

• a change is an edit to one object

• a change to something a composition produced goes back through the pipeline that produced it

single property, single resource, full stack and even whole cloud estate granularity of changes

Being able to apply changes in any granularity allows you to massively reduce risks, limit the blast radius of those changes and maintain focus, resulting in much less human error. Holistic Day-1-style rollouts on Day 2 create cognitive overload and more toil in cases where small changes are made, for example urgent fixes. Changing a single property of a single resource shouldn't feel like touching the whole infrastructure code.

• you can patch an object directly

• what a composition generates is regenerated rather than patched

• allows you to patch even single properties on single resources

• no need to see or provide irrelevant fields or resources at patch time

Being able to incrementally apply changes to single resources without dealing with irrelevant details is crucial for safety and speed. Blast radius control is a big and central topic in IaC. The less detail is needed to make an infrastructure change, the more confident you can be. Also, incremental changes are very common in many environments starting on Day 2. There are many situations where a patch is the simplest, quickest solution, and the IaC tool needs to support these.

the control plane watches the resources it declared

• detects every change and automatically absorbs it

• turns every change into versioned code

The term "drift" has been massively bloated in the classic IaC world. Since classic IaC tools are not able to automatically catch up on changes happening outside or their own idea of state, these changes have been declared as something very bad. Some tools even go so far that they automatically reconcile from their own state, undoing changes that happened outside of the tool. The reality in any reasonably complex environment is much more nuanced though. Multiple tools will be used, and changes of all kinds happen through all kinds of actors. Enforcing a single process and a single tool is often impossible, and also not reasonable. Drift as such should not be a concern - state and real infrastructure need to be always in perfect sync, with changes coming from both ends.

• drift is corrected back to the declaration, continuously, for resources somebody declared

• nothing else is corrected, because nothing else is watched

• and the change is corrected away rather than recorded, so what it was is not kept

automatically corrects drift both ways - to infrastructure and from infrastructure

Once drift has been automatically detected, it is crucial to also automatically correct it. It doesn't mean to restore infrastructure from the state one way - this would be only one of the possible strategies, and it shouldn't be applied uniformly. A more reality-friendly strategy is to accept changes coming from both ends. After all, when a change already happened in the target infrastructure, the state needs to be first updated, so there is no drift. After that, the user is free to decide if they want to undo the change. Or the change can be undone automatically according to some policy, but even after it has been registered and versioned.

• nothing is discovered

• adopting what already runs means writing a managed resource for each one and matching it by external name

• automatically discovers new resources

• automatically tracks and versions changes even on unmanaged resources

Automatic discovery of new resources makes use cases possible that are unthinkable or difficult to implement with the classic IaC tools. For example, you are able to analyse existing infrastructure immediately, even when there is no infrastructure code available. Also, patches on existing resources can be made quickly when these are discovered and turned into code - completely without the need to look at the rest of the infrastructure. On the other hand, classic IaC tools would send you through a manual process of importing existing resources that is prone to error. Automatic resource discovery also allows you to have a full inventory of existing resources for further processing entirely without manual work.

• a change made outside is corrected away, not absorbed

• what the change was is not kept

captures and versions every change that happens outside of the tool

Changes happen all the time outside of the IaC tool in a reasonably complex environment. It is a good idea to capture them automatically, not manually. Classic IaC tools require the user to update their state explicitly and manually, which inevitably leads to drift. On the other hand, state that is automatically up-to-date with the infrastructure makes changes predictable and reliable, and allows you to make explicit decisions to keep or to undo changes that happened outside of the tool.

• no versioned history of what changed and where it came from

• the cluster holds the current object, not the story

automatically versions all changes, no matter where they came from - be it from other tools, or even through ClickOps

When every change is automatically versioned, no matter where it came from, infrastructure is fully auditable. Also, versioning of changes allows you to review and to restore previous states at any time. Git is usually used for this in GitOps, but it comes at the price of code in Git being decoupled from state and from infrastructure. If the IaC tool takes care of change versioning, it is also one tool less to deal with in situations where change history needs to be analysed or used.

does not codify what it finds, because it does not look

• every change is automatically codified

• the user extracts resources or stacks exclusively as code

Users should communicate with the IaC tool exclusively through code. This requires every change to be codified automatically, no matter where it came from, so that the user can retrieve any current or past resource state as code. Classic IaC tools on the other hands Classic IaC tools, on the other hand, make users absorb their highly technical, low-level state format such as JSON and go through a complicated, manual procedure to turn it into actual infrastructure code.

• policy comes from the Kubernetes ecosystem rather than from Crossplane

• another admission controller to choose, install and operate

• offers TTL, auto-reconcile out of the box

• easy to extend with further policies

Automatic policy enforcement brings piece of mind and guardrails that prevent overspend, drift and toil.

• providers can be generated from a cloud API

• writing or operating one is Kubernetes controller work

• simple, uniform Plugin SDK with minimal boilerplate

• extensibility accessible to many engineers

• easily combined with AI code assistance

Extensibility should not be gated behind specialist expertise. When adding integrations is hard, it becomes a bottleneck: teams wait on scarce provider engineers and somebody else's roadmaps, product teams can’t self-serve, and platform evolution slows. Making extensions easy and safe unlocks broader contribution, faster iteration, and reduces dependency on scarce experts.

• building a provider is controller development

• the cycle runs through a cluster, which is slow and hard to debug

• many plugins can be built in under an hour with AI help

• safety enforced by uniform schema and lifecycle conventions

• agent does all the heavy lifting, making plugins simple

• no manual state machine or diff implementation required

Speed only matters if you can trust the result. Integrations that take weeks to build or are risky to validate get deferred, copied manually, or shipped with latent correctness issues. Rapid, safe plugin development means teams can add and evolve integrations continuously - without increasing operational risk or creating long-term maintenance debt.

• you run the cluster the control plane lives in, or pay a platform to run one for you

• either way the YAML is yours to write and keep in step

no code from you: nothing to write, and no repository to keep in step

A repository of infrastructure code is a second system to keep alive. Somebody writes it, somebody reviews it, and somebody keeps it in step with infrastructure that changes without asking it, which is work that never ends and never ships anything. Hosting the runs or the state elsewhere moves none of it. Not needing the repository at all is what removes it.

• the agent calls the Kubernetes API, which is a real operational surface: it creates, patches and deletes, and a controller acts on it

• it can only act on what somebody declared first, and there is nothing to ask about the rest of the estate

the agent calls the product's operations: discover, apply, patch, set a lifetime or auto-reconcile

An agent is only as useful as the surface it is handed. Given a codebase and a run pipeline, its work arrives as a code change that somebody still has to read, approve and release. Given the operations themselves, asking for a change and the change happening are one step, and the record of it is a version rather than a commit.