Introduce WG Checkpoint Restore #8508
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -3583,6 +3583,43 @@ workinggroups: | |||||||||||
| liaison: | ||||||||||||
| github: aojea | ||||||||||||
| name: Antonio Ojea | ||||||||||||
| - dir: wg-checkpoint-restore | ||||||||||||
| name: Checkpoint Restore | ||||||||||||
| mission_statement: > | ||||||||||||
| This working group aims to provide a central location for the community to discuss | ||||||||||||
| the integration of Checkpoint/Restore functionality into Kubernetes. | ||||||||||||
| charter_link: charter.md | ||||||||||||
|
There was a problem hiding this comment. is charter included into this PR? There was a problem hiding this comment. now it is, I didn't add it initially as the lifecycle document mentions that it is added later, but looking at the WG PRs it seems to be common to have a charter in the initial PR. |
||||||||||||
| stakeholder_sigs: | ||||||||||||
|
There was a problem hiding this comment. sig auth may have a big say in security of this whole restoration pipeline There was a problem hiding this comment. Thank you for pointing this out! Security is definitely an important topic that we need to discuss with sig-auth, both for the checkpoint API and the restoration pipeline. The following paper and master thesis describe our recent work on this topic: There was a problem hiding this comment. I added sig auth to the list of stakeholder sigs There was a problem hiding this comment. this showed up in the sig-auth meeting, we may have missed the discussion around this WG if this WG is contemplating taking state from a running pod / saving it / letting it be consumed on another node or from another pod or another namespace, then sig-auth is definitely interested in making sure the permissions model around that exists and is ~consistent with similar things Kubernetes does elsewhere (like PVC / snapshots) We're happy to consult on that, I'm not sure our awareness / involvement rises to the level of sponsoring the WG :) cc @kubernetes/sig-auth-leads There was a problem hiding this comment. This is a valuable initiative. The charter mentions that the scope includes checkpointing and restoring 'workloads' and providing 'guidance for developers on checkpoint-friendly app design.' Given this focus, it's essential for SIG Apps to be involved as a key stakeholder. There was a problem hiding this comment. @janetkuo This is a good idea, thank you so much for suggesting it! There was a problem hiding this comment. Yes, thanks @janetkuo. I added SIG Apps to the proposal. There was a problem hiding this comment. I agree with Janet here, but please make sure to show up and present the scope of this proposal to one of the future SIG-Apps calls. |
||||||||||||
| - API Machinery | ||||||||||||
| - Apps | ||||||||||||
| - Auth | ||||||||||||
| - CLI | ||||||||||||
| - Node | ||||||||||||
| - Scheduling | ||||||||||||
| label: checkpoint-restore | ||||||||||||
| leadership: | ||||||||||||
| chairs: | ||||||||||||
| - github: adrianreber | ||||||||||||
| name: Adrian Reber | ||||||||||||
| company: Red Hat | ||||||||||||
| email: [email protected] | ||||||||||||
| - github: haircommander | ||||||||||||
| name: Peter Hunt | ||||||||||||
| company: Red Hat | ||||||||||||
| email: [email protected] | ||||||||||||
| - github: rst0git | ||||||||||||
| name: Radostin Stoyanov | ||||||||||||
| company: University of Oxford | ||||||||||||
| email: [email protected] | ||||||||||||
| - github: viktoriaas | ||||||||||||
| name: Viktória Spišaková | ||||||||||||
| company: Masaryk University | ||||||||||||
| email: [email protected] | ||||||||||||
| meetings: [] | ||||||||||||
| contact: | ||||||||||||
| slack: wg-checkpoint-restore | ||||||||||||
| mailing_list: https://groups.google.com/forum/#!forum/kubernetes-wg-checkpoint-restore | ||||||||||||
|
There was a problem hiding this comment. This is missing the steering liason field, we discussed in the steering meeting today, you can add me.. Looking forward to being more directly helpful with @rst0git @viktoriaas and crew :-) There was a problem hiding this comment. Added, thanks
Suggested change
There was a problem hiding this comment. Did you forget to push the addition? There was a problem hiding this comment. @soltysh Thank you for your comment! Adrian is on vacation right now, but he will merge it when he returns. There was a problem hiding this comment. The mailing lists for all WGs/SIGs are part of our managed google groups, with a kubernetes.io domain. So this should be |
||||||||||||
| - dir: wg-data-protection | ||||||||||||
| name: Data Protection | ||||||||||||
| mission_statement: > | ||||||||||||
|
|
||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,38 @@ | ||||||||
| <!--- | ||||||||
| This is an autogenerated file! | ||||||||
| Please do not edit this file directly, but instead make changes to the | ||||||||
| sigs.yaml file in the project root. | ||||||||
| To understand how this file is generated, see https://git.k8s.io/community/generator/README.md | ||||||||
| ---> | ||||||||
| # Checkpoint Restore Working Group | ||||||||
|
|
||||||||
| This working group aims to provide a central location for the community to discuss the integration of Checkpoint/Restore functionality into Kubernetes. | ||||||||
|
|
||||||||
| The [charter](charter.md) defines the scope and governance of the Checkpoint Restore Working Group. | ||||||||
|
|
||||||||
| ## Stakeholder SIGs | ||||||||
| * [SIG API Machinery](/sig-api-machinery) | ||||||||
| * [SIG Apps](/sig-apps) | ||||||||
| * [SIG Auth](/sig-auth) | ||||||||
| * [SIG CLI](/sig-cli) | ||||||||
| * [SIG Node](/sig-node) | ||||||||
| * [SIG Scheduling](/sig-scheduling) | ||||||||
|
|
||||||||
|
|
||||||||
|
|
||||||||
| ## Organizers | ||||||||
|
|
||||||||
| * Adrian Reber (**[@adrianreber](https://github.com/adrianreber)**), Red Hat | ||||||||
| * Peter Hunt (**[@haircommander](https://github.com/haircommander)**), Red Hat | ||||||||
| * Radostin Stoyanov (**[@rst0git](https://github.com/rst0git)**), University of Oxford | ||||||||
| * Viktória Spišaková (**[@viktoriaas](https://github.com/viktoriaas)**), Masaryk University | ||||||||
|
|
||||||||
| ## Contact | ||||||||
| - Slack: [#wg-checkpoint-restore](https://kubernetes.slack.com/messages/wg-checkpoint-restore) | ||||||||
| - [Mailing list](https://groups.google.com/forum/#!forum/kubernetes-wg-checkpoint-restore) | ||||||||
| - [Open Community Issues/PRs](https://github.com/kubernetes/community/labels/wg%2Fcheckpoint-restore) | ||||||||
|
There was a problem hiding this comment. adding @BenTheElder as liaison to README too.
Suggested change
|
||||||||
| <!-- BEGIN CUSTOM CONTENT --> | ||||||||
|
|
||||||||
| <!-- END CUSTOM CONTENT --> | ||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,91 @@ | ||||||||||||||||||
|
|
||||||||||||||||||
| # WG Checkpoint Restore Charter | ||||||||||||||||||
|
|
||||||||||||||||||
| This charter adheres to the conventions described in the [Kubernetes Charter README] and uses | ||||||||||||||||||
| the Roles and Organization Management outlined in [sig-governance]. | ||||||||||||||||||
|
|
||||||||||||||||||
| ## Scope | ||||||||||||||||||
|
|
||||||||||||||||||
| The Checkpoint/Restore Working Group aims to solve the problem of transparently | ||||||||||||||||||
| checkpointing and restoring workloads in Kubernetes, a functionality discussed | ||||||||||||||||||
| for over five years. The group will deliver the design and implementation of | ||||||||||||||||||
|
There was a problem hiding this comment. Can you link to those discussions? |
||||||||||||||||||
| Checkpoint/Restore functionality in Kubernetes, serving as a central hub for | ||||||||||||||||||
|
There was a problem hiding this comment. Why it has to be a central part of Kubernetes, where multiple external solutions already exists? |
||||||||||||||||||
| community information and discussion. This initiative addresses a wide range of | ||||||||||||||||||
| problems, including fault tolerance, improved resource utilization, and | ||||||||||||||||||
| accelerated application startup times. | ||||||||||||||||||
|
There was a problem hiding this comment. This first thing that I'd like to point out is that there are 2 main use cases:
Which one this group is planning to cover? As I'm reading this document I'm seeing both used interchangeably which is very confusing. That's why I'd start with clearly drawing the line between the two and properly documenting which one of these two (or both) are you planning to tackle. |
||||||||||||||||||
|
|
||||||||||||||||||
| ### In scope | ||||||||||||||||||
|
|
||||||||||||||||||
| - Identify core Kubernetes checkpoint/restore use cases (e.g., live migration, | ||||||||||||||||||
| fault tolerance, debugging, snapshotting) and gather stakeholder requirements. | ||||||||||||||||||
| - Investigate and propose Kubernetes APIs for checkpoint/restore operations. | ||||||||||||||||||
| - Work with SIGs for the best integration of checkpoint/restore functionality | ||||||||||||||||||
| and APIs. | ||||||||||||||||||
| - Provide guidance for developers on checkpoint-friendly app design and | ||||||||||||||||||
|
There was a problem hiding this comment. there may be API needed to communicate between the app and API server that the checkopoint is requested AND/OR that the app is ready for checkpoint. Something that is beyond just guidance There was a problem hiding this comment. That is actually something we discussed how to do in containers for years now (outside of Kubernetes). But we never found the right way how to do this. We were looking at kernel interfaces or systemd interfaces because for many applications it could be helpful to free temporary memory to reduce checkpoint size or even drop confidential information. Also after restore it would be good to tell the application that maybe certain cryptographic values need to be reset or regenerated. I will try to include something mentioning this. Thanks. |
||||||||||||||||||
| recommendations for operators on feature management. | ||||||||||||||||||
| - Work closely with relevant upstream projects (CRI-O, containerd, CRIU, gVisor) | ||||||||||||||||||
| for alignment and integration. | ||||||||||||||||||
| - Revisit the existing implementations to find and remedy possible inefficiencies. | ||||||||||||||||||
| One example is the existing checkpoint archive format which has already been | ||||||||||||||||||
| identified as being a major source of slowdown. | ||||||||||||||||||
|
|
||||||||||||||||||
| ### Out of scope | ||||||||||||||||||
|
|
||||||||||||||||||
| - Not focused on general OS-level checkpointing outside Kubernetes | ||||||||||||||||||
| pods/containers. | ||||||||||||||||||
| - Will not dictate internal application checkpointing logic; focuses on | ||||||||||||||||||
| Kubernetes platform orchestration of *container/pod state. | ||||||||||||||||||
|
|
||||||||||||||||||
| ## Stakeholders | ||||||||||||||||||
|
|
||||||||||||||||||
| Stakeholders in this working group span multiple SIGs that own parts of the | ||||||||||||||||||
| code in core kubernetes components and addons. | ||||||||||||||||||
|
|
||||||||||||||||||
| - SIG CLI | ||||||||||||||||||
|
There was a problem hiding this comment. has there been outreach to these SIGs yet? I see some SIG node participants/leaders but not CLI yet for example There was a problem hiding this comment.
I have a simple prototype of a plugin for the There was a problem hiding this comment. OK, we should remove the SIG if they are not a stakeholder, or else the WG organizers should reach out to the SIGs. |
||||||||||||||||||
| - SIG API Machinery | ||||||||||||||||||
| - SIG Node | ||||||||||||||||||
| - SIG Scheduling | ||||||||||||||||||
| - SIG Auth | ||||||||||||||||||
| - SIG Apps | ||||||||||||||||||
|
|
||||||||||||||||||
| ## Deliverables | ||||||||||||||||||
|
|
||||||||||||||||||
| The list of deliverables include the following high level features: | ||||||||||||||||||
|
|
||||||||||||||||||
| - In the early stage, we mainly want to offer a well-defined location for the | ||||||||||||||||||
adrianreber marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||
| community to find information, ask questions, and discuss the next steps of | ||||||||||||||||||
| enabling checkpoint and restore in Kubernetes. | ||||||||||||||||||
|
|
||||||||||||||||||
| Later: | ||||||||||||||||||
|
|
||||||||||||||||||
| - Ability to checkpoint and restore a container using kubectl | ||||||||||||||||||
| - Ability to checkpoint and restore a pod using kubectl | ||||||||||||||||||
| - Integration of container/pod checkpointing in scheduling decisions | ||||||||||||||||||
|
There was a problem hiding this comment. Why pod checkpointing would have anything to do with scheduling? |
||||||||||||||||||
|
|
||||||||||||||||||
| ## Roles and Organization Management | ||||||||||||||||||
|
|
||||||||||||||||||
| This WG adheres to the Roles and Organization Management outlined in [wg-governance] | ||||||||||||||||||
| and opts-in to updates and modifications to [wg-governance]. | ||||||||||||||||||
|
|
||||||||||||||||||
| [wg-governance]: /committee-steering/governance/wg-governance.md | ||||||||||||||||||
|
|
||||||||||||||||||
| Additionally, the WG commits to: | ||||||||||||||||||
|
|
||||||||||||||||||
| - maintain a solid communication line between the Kubernetes groups and the | ||||||||||||||||||
| wider CNCF community | ||||||||||||||||||
| - submit a proposal to the KubeCon/CloudNativeCon maintainers track | ||||||||||||||||||
|
There was a problem hiding this comment. I have doubts if this incentivize the right behavior and will encourage people to build WG to get a slot in the kubecon There was a problem hiding this comment. I agree with Antonio, this particular line should be removed, it's sufficient what the previous point shows. |
||||||||||||||||||
|
|
||||||||||||||||||
| ## Timelines and Disbanding | ||||||||||||||||||
|
|
||||||||||||||||||
| As a first mandate, the WG will define a roadmap and tasks in the first quarter | ||||||||||||||||||
| of operation. | ||||||||||||||||||
|
|
||||||||||||||||||
| After that the WG will distribute the different tasks to different community | ||||||||||||||||||
| members to define possible APIs and how it can be integrated in Kubernetes. | ||||||||||||||||||
|
Comment on lines
+80
to
+84
There was a problem hiding this comment. we may need some clarification on this paragraph, for readers unfamiliar with Kubernetes governance, the current wording ("WG will define a roadmap," "distribute tasks," "define possible APIs") could give the impression that the WG is an authoritative body with the power to approve APIs and dictate roadmaps for the project. As we know, a WG's primary role is to serve as a forum for cross-SIG collaboration, and SIGs are the ultimate authority for code and API changes. To avoid this potential confusion, could we rephrase this section to better reflect the WG's role as a facilitator and a place for proposing ideas to the SIGs? what about something like:
Suggested change
|
||||||||||||||||||
|
|
||||||||||||||||||
| Achieving the aforementioned deliverables, also mentioned in the `In Scope` | ||||||||||||||||||
| section, will allow us to decide when to disband this WG. There is no | ||||||||||||||||||
| expectations that the Working Group will be converted into a SIG long term. | ||||||||||||||||||
|
|
||||||||||||||||||
| [sig-governance]: https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance.md | ||||||||||||||||||
| [Kubernetes Charter README]: https://github.com/kubernetes/community/blob/master/committee-steering/governance/README.md | ||||||||||||||||||
Uh oh!
There was an error while loading. Please reload this page.