Architecture¶
Status
The Foreman and SmartProxy controllers are implemented as an MVP. The frontend is part of Foreman and runs in the web pods, and the recurring tasks run as CronJobs. Release and the Katello kinds are target design. Rows and sections below say which parts are planned.
Overview¶
The operator follows the standard Kubernetes operator pattern. Controllers watch custom resources and create native workloads directly, with owner references, without rendering through Helm.
flowchart LR
Users[Users and API clients] --> Edge[Route, LoadBalancer, or TLSRoute, TLS passthrough]
Build[Hosts being built] -->|HTTP unattended| Edge
Edge --> Frontend
subgraph WebPod[Foreman web pod]
Frontend[httpd, mutual TLS] -->|Unix socket| Web[Puma]
end
Dynflow[Dynflow] --> PG[(PostgreSQL)]
Dynflow --> Valkey[(Valkey)]
Web --> PG
Web --> Valkey
Web <-->|mutual TLS| Prov[SmartProxy: Provisioning]
Web <-->|mutual TLS| Exec[SmartProxy: Execution]
Exec -->|SSH| Managed[Managed hosts]
Prov -->|Multus| Net[Provisioning networks]
Build -->|DHCP, TFTP, templates| Net
Operator startup¶
At startup, the operator checks which optional APIs the cluster serves and only watches kinds that exist:
| Capability | Detected through | Used for |
|---|---|---|
| cert-manager | cert-manager.io/v1 Certificate |
Every tls.issuerRef |
| Gateway API | gateway.networking.k8s.io/v1 TLSRoute and HTTPRoute |
Exposure TLSRoute |
| OpenShift | route.openshift.io/v1 Route |
Exposure Route and Auto, no fsGroup in pod security contexts |
A resource that needs a missing capability fails with a reconcile error. APIs installed after the operator started are only seen after a restart.
| Flag | Default | Description |
|---|---|---|
--default-image-tag |
develop |
Tag of the component images when a resource names neither tag nor digest. The chart sets it to its appVersion |
--watch-namespace |
empty | Restricts the operator to one namespace. Empty watches all namespaces |
--leader-elect |
false |
Enables leader election |
--metrics-secure |
false |
Serves metrics over HTTPS with authentication and authorization |
CRD Relationships¶
flowchart TD
SmartProxy -->|foremanRef| Foreman
Foreman -->|database.cluster| CNPG[CloudNativePG Cluster]
Foreman -->|tls.issuerRef| Internal[cert-manager internal CA]
SmartProxy -->|tls.issuerRef| Internal
Foreman -->|frontend.tls.clientCA| Internal
Foreman -->|frontend.tls.server| Public[cert-manager public issuer]
SmartProxy -->|networks.attachment| NAD[NetworkAttachmentDefinition]
Workloads¶
| Component | Workload | Replicas | Process | State |
|---|---|---|---|---|
| Foreman web | Deployment | 2 by default | Container foreman: Rails with Puma on a Unix socket. Container httpd: Apache with mod_ssl on ports 8443 and 8080 |
Implemented |
| Dynflow orchestrator | Deployment, Recreate | 1 | Sidekiq with the orchestrator configuration | Implemented |
| Dynflow worker | Deployment | 1 by default | Sidekiq with the worker configuration | Implemented |
| Dynflow hosts queue | Deployment | 0 by default | Sidekiq with the hosts queue configuration | Implemented |
| Foreman preflight | Job | n/a | Reads the Foreman version of a new image, see Upgrades | Implemented |
| Foreman migration | Job | n/a | db:migrate, db:seed, and upgrade:run |
Implemented |
| Smart Proxy | Deployment, Recreate | 1 | See Smart Proxy | Implemented for Provisioning and Execution |
| Recurring tasks | CronJob | n/a | rails cron:hourly, daily, weekly, monthly |
Implemented |
The process split follows the foremanctl deployment. Apache runs as a sidecar of Puma rather than as a separate Deployment, so Puma has no TCP listener and no other pod can forge the client certificate headers. Mutual TLS between a separate frontend and Puma would need extra certificates and still leave a port open. The trade-off is that frontend and web scale 1:1 and a frontend change restarts the web pods. See Traffic Flow. Puma workers are derived from the memory limit of the web container, about 1.5 GiB per worker, between 1 and 16.
Foreman core uses Dynflow as its ActiveJob backend, independent of Katello, for example for report templates, notifications, and event subscribers. The orchestrator and one worker are therefore part of every deployment. The hosts queue is only needed for Remote Execution, so it defaults to zero replicas.
Deliberately not CRDs¶
- Certificates are cert-manager resources.
- Plugins are a list on
Foreman. - DHCP, TFTP, and DNS are features of a
SmartProxy, because they share a pod and its state. - Subnets, domains, and hosts stay in Foreman. A second source of truth would conflict with Foreman's own data.
Patterns¶
- One builder per child object.
- Configuration through mounted files, generated by the operator, not entrypoint scripts.
- A checksum of the rendered configuration in the pod template annotation
platform.theforeman.org/config-checksumtriggers rollouts. It includes the data of the certificate Secrets, and healthy resources are reconciled every 10 minutes, so a certificate renewed by cert-manager rolls the pods out within that interval. - The annotation
platform.theforeman.org/paused: "true"suspends reconciliation of one object. - Objects that must survive the custom resource, such as
<name>-secretsof a Foreman and the PVC of a Smart Proxy, have no owner reference. - All pods disable service links.
- Read-only root filesystems in every container. Writable paths are volumes:
/tmp(alsoHOME), the Railstmpdirectory, the Puma socket directory,/run/httpd, and the data volume of the Smart Proxy.