Skip to content

Middleware Wiring Improvements #8528

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jun 26, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/docs/01-ibc/03-apps/01-apps.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,23 @@ encoded version into each handhshake call as necessary.

ICS20 currently implements basic string matching with a single supported version.

### ICS4Wrapper

The IBC application interacts with core IBC through the `ICS4Wrapper` interface for any application-initiated actions like: `SendPacket` and `WriteAcknowledgement`. This may be directly the IBCChannelKeeper or a middleware that sits between the application and the IBC ChannelKeeper.

If the application is being wired with a custom middleware, the application **must** have its ICS4Wrapper set to the middleware directly above it on the stack through the following call:

```go
// SetICS4Wrapper sets the ICS4Wrapper. This function may be used after
// the module's initialization to set the middleware which is above this
// module in the IBC application stack.
// The ICS4Wrapper **must** be used for sending packets and writing acknowledgements
// to ensure that the middleware can intercept and process these calls.
// Do not use the channel keeper directly to send packets or write acknowledgements
// as this will bypass the middleware.
SetICS4Wrapper(wrapper ICS4Wrapper)
```

### Custom Packets

Modules connected by a channel must agree on what application data they are sending over the
Expand Down
33 changes: 29 additions & 4 deletions docs/docs/01-ibc/04-middleware/02-develop.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ The interfaces a middleware must implement are found [here](https://github.com/c
type Middleware interface {
IBCModule // middleware has access to an underlying application which may be wrapped by more middleware
ICS4Wrapper // middleware has access to ICS4Wrapper which may be core IBC Channel Handler or a higher-level middleware that wraps this middleware.

// SetUnderlyingModule sets the underlying IBC module. This function may be used after
// the middleware's initialization to set the ibc module which is below this middleware.
SetUnderlyingApplication(IBCModule)
}
```

Expand All @@ -42,14 +46,12 @@ An `IBCMiddleware` struct implementing the `Middleware` interface, can be define
// IBCMiddleware implements the ICS26 callbacks and ICS4Wrapper for the fee middleware given the
// fee keeper and the underlying application.
type IBCMiddleware struct {
app porttypes.IBCModule
keeper keeper.Keeper
keeper *keeper.Keeper
}

// NewIBCMiddleware creates a new IBCMiddleware given the keeper and underlying application
func NewIBCMiddleware(app porttypes.IBCModule, k keeper.Keeper) IBCMiddleware {
func NewIBCMiddleware(k *keeper.Keeper) IBCMiddleware {
return IBCMiddleware{
app: app,
keeper: k,
}
}
Expand Down Expand Up @@ -476,3 +478,26 @@ func GetAppVersion(
```

See [here](https://github.com/cosmos/ibc-go/blob/v7.0.0/modules/apps/29-fee/keeper/relay.go#L58-L74) an example implementation of this function for the ICS-29 Fee Middleware module.

## Wiring Interface Requirements

Middleware must also implement the following functions so that they can be called in the stack builder in order to correctly wire the application stack together: `SetUnderlyingApplication` and `SetICS4Wrapper`.

```go
// SetUnderlyingModule sets the underlying IBC module. This function may be used after
// the middleware's initialization to set the ibc module which is below this middleware.
SetUnderlyingApplication(IBCModule)

// SetICS4Wrapper sets the ICS4Wrapper. This function may be used after
// the module's initialization to set the middleware which is above this
// module in the IBC application stack.
// The ICS4Wrapper **must** be used for sending packets and writing acknowledgements
// to ensure that the middleware can intercept and process these calls.
// Do not use the channel keeper directly to send packets or write acknowledgements
// as this will bypass the middleware.
SetICS4Wrapper(wrapper ICS4Wrapper)
```

The middleware itself should have access to the `underlying app` (note this may be a base app or an application wrapped by layers of lower-level middleware(s)) and access to the higher layer `ICS4wrapper`. The `underlying app` gets called during the relayer initiated actions: `recvPacket`, `acknowledgePacket`, and `timeoutPacket`. The `ics4Wrapper` gets called on user-initiated actions like `sendPacket` and `writeAcknowledgement`.

The functions above are used by the `StackBuilder` during application setup to wire the stack correctly. The stack must be wired first and have all of the wrappers and applications set correctly before transaction execution starts and packet processing begins.
28 changes: 22 additions & 6 deletions docs/docs/01-ibc/04-middleware/03-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@ The order of middleware **matters**, function calls from IBC to the application

// middleware 1 and middleware 3 are stateful middleware,
// perhaps implementing separate sdk.Msg and Handlers
mw1Keeper := mw1.NewKeeper(storeKey1, ..., ics4Wrapper: channelKeeper, ...) // in stack 1 & 3
// NOTE: NewKeeper returns a pointer so that we can modify
// the keepers later after initialization
// They are all initialized to use the channelKeeper directly at the start
mw1Keeper := mw1.NewKeeper(storeKey1, ..., channelKeeper) // in stack 1 & 3
// middleware 2 is stateless
mw3Keeper1 := mw3.NewKeeper(storeKey3,..., ics4Wrapper: mw1Keeper, ...) // in stack 1
mw3Keeper2 := mw3.NewKeeper(storeKey3,..., ics4Wrapper: channelKeeper, ...) // in stack 2
mw3Keeper1 := mw3.NewKeeper(storeKey3,..., channelKeeper) // in stack 1
mw3Keeper2 := mw3.NewKeeper(storeKey3,..., channelKeeper) // in stack 2

// Only create App Module **once** and register in app module
// if the module maintains independent state and/or processes sdk.Msgs
Expand All @@ -55,13 +58,26 @@ customIBCModule1 := custom.NewIBCModule(customKeeper1, "portCustom1")
customIBCModule2 := custom.NewIBCModule(customKeeper2, "portCustom2")

// create IBC stacks by combining middleware with base application
// IBC Stack builders are initialized with the IBC ChannelKeeper which is the top-level ICS4Wrapper
// NOTE: since middleware2 is stateless it does not require a Keeper
// stack 1 contains mw1 -> mw3 -> transfer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from looking at this, although you specify the order I still don't think it is easy to understand conceptually and why one middleware should be before another in the ordering. Just having more examples doesn't really explain how you reason about the order of the stack. I think it is fine to have the mock examples with mw1 etc, but I think it would be useful to give more detail on how to reason about the underlying order. Concretely we have 3 middlewares in the repo and the assumption is users should expect to have them all for transfer so I think we should concretely relate to this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good point. The code itself helps reason about what order you actually end up getting, but not necessarily which order they ought to be in.

It's hard to get too concrete on this, since it depends on which middlewares you use, but we can certainly use the ones in the repo as an example. We should def do that.

I think we could add a section on middleware ordering where we go through how the ordering works (i.e. how the ordering relates to the actual flow of a packet on send and receive) and use our own middlewares as an example to show how we decided on the order of those.

(@womensrights: Separately from this, the docs for each middleware should probably also make it clear if they have any place in the stack they should be - this is particularly relevant for rate limiting!)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes agreed, like there are likely some heuristics that could help you reason through ordering i.e. should this middleware act like a gate for any further processing (e.g. rate limiting) or should this middleware always be last to execute... just from the top of my head

and yes I think we should just be very concrete with what we have under our maintenance so reasoning should be really clear for pfm, rate limiting and callbacks in a stack

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also thinking about this more, it seems we are going to have multiple ways to wire middleware if you use the stack builder or not, I think this is going to be messy and confusing and there should only be one recommended way

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so perhaps this is just a docs thing as people could still choose to use the old way, but I think we only show the new way from v11 docs onwards

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have good context anymore on exactly what order we should be prescribing. Especially in the context of the new middlewares in the repo.

We only are showing the stack builder approach in the docs from now on and only recommending this approach. Though i imagine some people will still use the old way.

I tried addressing this, but feel im missing the information i need at this point in terms of exactly what you want here. Since its not really in scope for the stack builder approach, Id prefer to merge this and have this comment be a general improvement of the middleware docs that we should tackle separately

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll merge this now, but sounds like there are some doc improvements needed @womensrights

stack1 := mw1.NewIBCMiddleware(mw3.NewIBCMiddleware(transferIBCModule, mw3Keeper1), mw1Keeper)
stack1 := porttypes.NewStackBuilder(ibcChannelKeeper).
Base(transferIBCModule).
Next(mw3).
Next(mw1).
Build()
// stack 2 contains mw3 -> mw2 -> custom1
stack2 := mw3.NewIBCMiddleware(mw2.NewIBCMiddleware(customIBCModule1), mw3Keeper2)
stack2 := porttypes.NewStackBuilder(ibcChannelKeeper).
Base(customIBCModule1).
Next(mw2).
Next(mw3).
Build()
// stack 3 contains mw2 -> mw1 -> custom2
stack3 := mw2.NewIBCMiddleware(mw1.NewIBCMiddleware(customIBCModule2, mw1Keeper))
stack3 := porttypes.NewStackBuilder(ibcChannelKeeper).
Base(customIBCModule2).
Next(mw1).
Next(mw2).
Build()

// associate each stack with the moduleName provided by the underlying Keeper
ibcRouter := porttypes.NewRouter()
Expand Down
148 changes: 148 additions & 0 deletions docs/docs/05-migrations/15-support-stackbuilder.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
---
title: Support the new StackBuilder primitive for Wiring Middlewares in the chain application
sidebar_label: Support StackBuilder Wiring
sidebar_position: 1
slug: /migrations/support-stackbuilder
---

# Migration for Chains wishing to use StackBuilder

The StackBuilder struct is a new primitive for wiring middleware in a simpler and less error-prone manner. It is not a breaking change thus the existing method of wiring middleware still works, though it is highly recommended to transition to the new wiring method.

Refer to the [integration guide](../01-ibc/04-middleware/03-integration.md) to understand how to use this new middleware to improve middleware wiring in the chain application setup.

# Migrations for Application Developers

In order to be wired with the new StackBuilder primitive, applications and middlewares must implement new methods as part of their respective interfaces.

IBC Applications must implement a new `SetICS4Wrapper` which will set the `ICS4Wrapper` through which the application will call `SendPacket` and `WriteAcknowledgement`. It is recommended that IBC applications are initialized first with the IBC ChannelKeeper directly, and then modified with a middleware ICS4Wrapper during the stack wiring.

```go
// SetICS4Wrapper sets the ICS4Wrapper. This function may be used after
// the module's initialization to set the middleware which is above this
// module in the IBC application stack.
// The ICS4Wrapper **must** be used for sending packets and writing acknowledgements
// to ensure that the middleware can intercept and process these calls.
// Do not use the channel keeper directly to send packets or write acknowledgements
// as this will bypass the middleware.
SetICS4Wrapper(wrapper ICS4Wrapper)
```

Many applications have a stateful keeper that executes the logic for sending packets and writing acknowledgements. In this case, the keeper in the application must be a **pointer** reference so that it can be modified in place after initialization.

The initialization should be modified to no longer take in an addition `ics4Wrapper` as this gets modified later by `SetICS4Wrapper`. The constructor function must also return a **pointer** reference so that it may be modified in-place by the stack builder.

Below is an example IBCModule that supports the stack builder wiring.

E.g.

```go
type IBCModule struct {
keeper *keeper.Keeper
}

// NewIBCModule creates a new IBCModule given the keeper
func NewIBCModule(k *keeper.Keeper) *IBCModule {
return &IBCModule{
keeper: k,
}
}

// SetICS4Wrapper sets the ICS4Wrapper. This function may be used after
// the module's initialization to set the middleware which is above this
// module in the IBC application stack.
func (im IBCModule) SetICS4Wrapper(wrapper porttypes.ICS4Wrapper) {
if wrapper == nil {
panic("ICS4Wrapper cannot be nil")
}

im.keeper.WithICS4Wrapper(wrapper)
}

/// Keeper file that has ICS4Wrapper internal to its own struct

// Keeper defines the IBC fungible transfer keeper
type Keeper struct {
...
ics4Wrapper porttypes.ICS4Wrapper

// Keeper is initialized with ICS4Wrapper
// being equal to the top-level channelKeeper
// this can be changed by calling WithICS4Wrapper
// with a different middleware ICS4Wrapper
channelKeeper types.ChannelKeeper
...
}

// WithICS4Wrapper sets the ICS4Wrapper. This function may be used after
// the keepers creation to set the middleware which is above this module
// in the IBC application stack.
func (k *Keeper) WithICS4Wrapper(wrapper porttypes.ICS4Wrapper) {
k.ics4Wrapper = wrapper
}
```

# Migration for Middleware Developers

Since Middleware is itself implement the IBC application interface, it must also implement `SetICS4Wrapper` in the same way as IBC applications.

Additionally, IBC Middleware has an underlying IBC application that it calls into as well. Previously this application would be set in the middleware upon construction. With the stack builder primitive, the application is only set during upon calling `stack.Build()`. Thus, middleware is additionally responsible for implementing the new method: `SetUnderlyingApplication`:

```go
// SetUnderlyingModule sets the underlying IBC module. This function may be used after
// the middleware's initialization to set the ibc module which is below this middleware.
SetUnderlyingApplication(IBCModule)
```

The initialization should not include the ICS4Wrapper and application as this gets set later. The constructor function for Middlewares **must** be modified to return a **pointer** reference so that it can be modified in place by the stack builder.

Below is an example middleware setup:

```go
// IBCMiddleware implements the ICS26 callbacks
type IBCMiddleware struct {
app porttypes.PacketUnmarshalerModule
ics4Wrapper porttypes.ICS4Wrapper

// this is a stateful middleware with its own internal keeper
mwKeeper *keeper.MiddlewareKeeper

// this is a middleware specific field
mwField any
}

// NewIBCMiddleware creates a new IBCMiddleware given the keeper and underlying application.
// NOTE: It **must** return a pointer reference so it can be
// modified in place by the stack builder
// NOTE: We do not pass in the underlying app and ICS4Wrapper here as this happens later
func NewIBCMiddleware(
mwKeeper *keeper.MiddlewareKeeper, mwField any,
) *IBCMiddleware {
return &IBCMiddleware{
mwKeeper: mwKeeper,
mwField, mwField,
}
}

// SetICS4Wrapper sets the ICS4Wrapper. This function may be used after the
// middleware's creation to set the middleware which is above this module in
// the IBC application stack.
func (im *IBCMiddleware) SetICS4Wrapper(wrapper porttypes.ICS4Wrapper) {
if wrapper == nil {
panic("ICS4Wrapper cannot be nil")
}
im.mwKeeper.WithICS4Wrapper(wrapper)
}

// SetUnderlyingApplication sets the underlying IBC module. This function may be used after
// the middleware's creation to set the ibc module which is below this middleware.
func (im *IBCMiddleware) SetUnderlyingApplication(app porttypes.IBCModule) {
if app == nil {
panic(errors.New("underlying application cannot be nil"))
}
if im.app != nil {
panic(errors.New("underlying application already set"))
}
im.app = app
}
```
3 changes: 3 additions & 0 deletions e2e/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ module github.com/cosmos/ibc-go/e2e

go 1.24.3

// TODO: Remove when v11 release of interchaintest is available (that is where this one is coming from)
replace github.com/cosmos/interchain-security/v7 => github.com/cosmos/interchain-security/v7 v7.0.0-20250622154438-73c73cf686e5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to add this, because interchaintest pulls in this, and the changes here break it :/
The PR that has the commit referenced in the replace: cosmos/interchain-security#2622


replace (
github.com/cosmos/ibc-go/modules/light-clients/08-wasm/v10 => ../modules/light-clients/08-wasm
// uncomment to use the local version of ibc-go, you will need to run `go mod tidy` in e2e directory.
Expand Down
4 changes: 2 additions & 2 deletions e2e/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -856,8 +856,8 @@ github.com/cosmos/iavl v1.2.4 h1:IHUrG8dkyueKEY72y92jajrizbkZKPZbMmG14QzsEkw=
github.com/cosmos/iavl v1.2.4/go.mod h1:GiM43q0pB+uG53mLxLDzimxM9l/5N9UuSY3/D0huuVw=
github.com/cosmos/ics23/go v0.11.0 h1:jk5skjT0TqX5e5QJbEnwXIS2yI2vnmLOgpQPeM5RtnU=
github.com/cosmos/ics23/go v0.11.0/go.mod h1:A8OjxPE67hHST4Icw94hOxxFEJMBG031xIGF/JHNIY0=
github.com/cosmos/interchain-security/v7 v7.0.0-20250408210344-06e0dc6bf6d6 h1:SzJ/+uqrTsJmI+f/GqPdC4lGxgDQKYvtRCMXFdJljNM=
github.com/cosmos/interchain-security/v7 v7.0.0-20250408210344-06e0dc6bf6d6/go.mod h1:W7JHsNaZ5XoH88cKT+wuCRsXkx/Fcn2kEwzpeGdJBxI=
github.com/cosmos/interchain-security/v7 v7.0.0-20250622154438-73c73cf686e5 h1:6LnlaeVk/wHPicQG8NqElL1F1FDVGEl7xF0JXGGhgEs=
github.com/cosmos/interchain-security/v7 v7.0.0-20250622154438-73c73cf686e5/go.mod h1:9EIcx4CzKt/5/2KHtniyzt7Kz8Wgk6fdvyr+AFIUGHc=
github.com/cosmos/interchaintest/v10 v10.0.0 h1:DEsXOS10x191Q3EU4RkOnyqahGCTnLaBGEN//C2MvUQ=
github.com/cosmos/interchaintest/v10 v10.0.0/go.mod h1:caS4BRkAg8NkiZ8BsHEzjNBibt2OVdTctW5Ezz+Jqxs=
github.com/cosmos/ledger-cosmos-go v0.14.0 h1:WfCHricT3rPbkPSVKRH+L4fQGKYHuGOK9Edpel8TYpE=
Expand Down
Loading
Loading