You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
Kubernetes Borg/Omega history topic 8: Declarative configuration and Apply. Inside Google, the most used configuration approach for Borg is the Turing-complete Borg Configuration Language (BCL). You can see a snippet of BCL on slide 7 in this deck: http://www.inf.ed.ac.uk/teaching/courses/exc/slides/Wilkes.pdf …
Millions of lines of BCL have been written. A fair amount of BCL was devoted to configuring application command-line flags, which was the most common way to figure server binaries, which is crazy IMO, but the practice sadly carried over to Kubernetes components
BCL was evaluated and instantiated using the borgcfg CLI, which supports commands like up, down, and update. Logic to diff and merge, perform rolling updates, and otherwise update the live state was embedded in the tool. Logic for common generation functions was written in BCL
This created a monolithic configuration and tool ecosystem. Even frameworks like mapreduce and services on top of Borg like BorgCron had to use BCL and borgcfg to interact with Borg. Getting-started tools generated BCL
A Python-based language was later developed, also. It interfaced with the update logic via a protobuf that wasn't quite the same as Borgmaster's. Other languages, such as Ruby, weren't used in Google. Several new Borg config languages were developed, but none were approved
Not specifically developed for Borg use, https://jsonnet.org/articles/design.html … and https://github.com/cuelang/cue/ were inspired by BCL. http://aurora.apache.org/documentation/latest/reference/configuration-templating/ … and https://github.com/stripe/skycfg were inspired by the Python language.
Borgcfg didn't provide configuration packages. Shared templates were unversioned and directly imported from their homes in the monorepo, which inflicted churn on their consumers. There were also no "stacks" or lifecycle directives, so a number of imperative updates were needed
In Kubernetes, we wanted to decouple configuration authoring and generation from updates to the desired state via the API, so that users could express configuration using languages and tools familiar to them: Jinja, Python, Ruby, Javascript, Terraform, Ansible, whatever
I wrote about this in http://prs.k8s.io/1007 . I also felt it needed to be possible for automation to write directly to the API and not need to update some arbitrary configuration language. To do that, we needed to be able to merge user intent and automated changes
My initial proposal, in http://issues.k8s.io/1178 , was to maintain and merge 2 separate layers of desired state in the server. Resistance to that idea led to my client-side Apply proposal in http://issues.k8s.io/1702 . We're finally getting server-side apply:https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/0006-apply.md …
A gotcha we ran into early in the Apply implementation was complex schema topology. Merging 2 flat maps is easy, but we unfortunately had associative lists: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#lists-of-named-subobjects-preferred-over-maps …. And also sets and undiscriminated unions (being addressed: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/20190325-unions.md …).
Strategic merge patch was developed so that we could diff and merge two objects containing associative lists (non-ordinal lists with index keys in values of fields within list elements): https://github.com/kubernetes/community/blob/master/contributors/devel/sig-api-machinery/strategic-merge-patch.md …
I wrote an overview of the motivation and principles for the configuration design in https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/declarative-application-management.md …. The original draft of that also contained sketches of what became https://github.com/kubernetes-sigs/application … andhttps://github.com/kubernetes-sigs/kustomize …
Whereas Apply facilitates collaborative config authoring between humans and machines (thanks to @originalavalamp for that description), kustomize enables collaboration among humans, by facilitating modification of unchanged base prototype/seed configurations.
The declarative API, Apply, and kustomize facilitate maintaining configuration as YAML or JSON or proto, amenable to manipulation by tools, rather than as YAML marked up with macros, complex configuration languages, or scripts written in general-purpose programming language.
On one hand, the ~100 tools that have been developed show that the decoupling of config format and the API has worked. OTOH, it shows there are still gaps. With work like diff and dry run (https://github.com/kubernetes/enhancements/pull/893 …) and prune (https://github.com/kubernetes/enhancements/pull/810 …), we're working to close them
A list of tools can be found here: https://docs.google.com/spreadsheets/d/1FCgqz1Ci7_VCz_wdh8vBitZ3giBtac_H8SBw4uxnrsE/edit#gid=0 …. I just added another 20 or so that I've seen.
This thread is already the longest yet, so I'll start another later with configuration terminology: declarative vs intent, macros vs config languages, packages vs stacks, prototypes vs templates, whitebox vs blackbox, overlays, lifecycle directives, etc.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
