feat: Integrate listener operator

CHANGELOG.md

@@ -7,10 +7,12 @@ All notable changes to this project will be documented in this file.
 ### Added
 
 - Add rolling upgrade support for upgrades between NiFi 2 versions ([#771]).
+- Added listener support for Nifi ([#784]).
 - Adds new telemetry CLI arguments and environment variables ([#782]).
   - Use `--file-log-max-files` (or `FILE_LOG_MAX_FILES`) to limit the number of log files kept.
   - Use `--file-log-rotation-period` (or `FILE_LOG_ROTATION_PERIOD`) to configure the frequency of rotation.
   - Use `--console-log-format` (or `CONSOLE_LOG_FORMAT`) to set the format to `plain` (default) or `json`.
+- Add test for Apache Iceberg integration ([#785]).
 
 ### Changed
 
@@ -39,7 +41,9 @@ All notable changes to this project will be documented in this file.
 [#774]: https://github.com/stackabletech/nifi-operator/pull/774
 [#776]: https://github.com/stackabletech/nifi-operator/pull/776
 [#782]: https://github.com/stackabletech/nifi-operator/pull/782
+[#785]: https://github.com/stackabletech/nifi-operator/pull/785
 [#787]: https://github.com/stackabletech/nifi-operator/pull/787
+[#784]: https://github.com/stackabletech/nifi-operator/pull/784
 [#789]: https://github.com/stackabletech/nifi-operator/pull/789
 
 ## [25.3.0] - 2025-03-21

deploy/helm/nifi-operator/crds/crds.yaml

@@ -94,20 +94,6 @@ spec:
                           description: Allow all proxy hosts by turning off host header validation. See <https://github.com/stackabletech/docker-images/pull/694>
                           type: boolean
                       type: object
-                    listenerClass:
-                      default: cluster-internal
-                      description: |-
-                        This field controls which type of Service the Operator creates for this NifiCluster:
-
-                        * cluster-internal: Use a ClusterIP service
-
-                        * external-unstable: Use a NodePort service
-
-                        This is a temporary solution with the goal to keep yaml manifests forward compatible. In the future, this setting will control which [ListenerClass](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass.html) will be used to expose the service, and ListenerClass names will stay the same, allowing for a non-breaking change.
-                      enum:
-                        - cluster-internal
-                        - external-unstable
-                      type: string
                     sensitiveProperties:
                       description: These settings configure the encryption of sensitive properties in NiFi processors. NiFi supports encrypting sensitive properties in processors as they are written to disk. You can configure the encryption algorithm and the key to use. You can also let the operator generate an encryption key for you.
                       properties:
@@ -274,6 +260,10 @@ spec:
                           description: Time period Pods have to gracefully shut down, e.g. `30m`, `1h` or `2d`. Consult the operator documentation for details.
                           nullable: true
                           type: string
+                        listenerClass:
+                          description: This field controls which [ListenerClass](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass.html) is used to expose the webserver.
+                          nullable: true
+                          type: string
                         logging:
                           default:
                             containers: {}
@@ -761,6 +751,10 @@ spec:
                                 description: Time period Pods have to gracefully shut down, e.g. `30m`, `1h` or `2d`. Consult the operator documentation for details.
                                 nullable: true
                                 type: string
+                              listenerClass:
+                                description: This field controls which [ListenerClass](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass.html) is used to expose the webserver.
+                                nullable: true
+                                type: string
                               logging:
                                 default:
                                   containers: {}

deploy/helm/nifi-operator/templates/roles.yaml

@@ -90,6 +90,12 @@ rules:
     verbs:
       - create
       - patch
+  - apiGroups:
+      - listeners.stackable.tech
+    resources:
+      - listeners
+    verbs:
+      - get
   - apiGroups:
       - {{ include "operator.name" . }}.stackable.tech
     resources:

docs/modules/nifi/pages/usage_guide/listenerclass.adoc

@@ -1,19 +1,14 @@
 = Service exposition with ListenerClasses
 :description: Configure Apache NiFi service exposure with cluster-internal or external-unstable listener classes.
 
-Apache NiFi offers a web UI and an API.
-The Operator deploys a service called `<name>` (where `<name>` is the name of the NifiCluster) through which NiFi can be reached.
-
-This service can have either the `cluster-internal` or `external-unstable` type.
-`external-stable` is not supported for NiFi at the moment.
-Read more about the types in the xref:concepts:service-exposition.adoc[service exposition] documentation at platform level.
-
-This is how the listener class is configured:
+The operator deploys a xref:listener-operator:listener.adoc[Listener] for the Node pod.
+The listener defaults to only being accessible from within the Kubernetes cluster, but this can be changed by setting `.spec.nodes.config.listenerClass`:
 
 [source,yaml]
 ----
 spec:
-  clusterConfig:
-    listenerClass: cluster-internal  # <1>
+  nodes:
+    config:
+      listenerClass: external-unstable  # <1>
 ----
-<1> The default `cluster-internal` setting.
+<1> Specify one of `external-stable`, `external-unstable`, `cluster-internal` (the default setting is `cluster-internal`).

docs/modules/nifi/pages/usage_guide/updating.adoc

@@ -24,7 +24,7 @@ NiFi clusters cannot be upgraded or downgraded in a rolling fashion due to a lim
 When upgrading between NiFi 1 versions or from NiFi 1 to NiFi 2, any change to the NiFi version in the CRD triggers a full cluster restart with brief downtime.
 However, the Stackable image version can be updated in a rolling manner, provided the NiFi version remains unchanged.
 
-For upgrades between NiFi 2 versions, e.g. from `2.0.0` to `2.2.0`, rolling upgrades are supported.
+For upgrades between NiFi 2 versions, e.g. from `2.0.0` to `2.4.0`, rolling upgrades are supported.
 ====
 
 == NiFi 2.0.0

docs/modules/nifi/pages/usage_guide/writing-to-iceberg-tables.adoc

@@ -2,13 +2,27 @@
 :description: Write to Apache Iceberg tables in NiFi using the PutIceberg processor. Supports integration with S3 and Hive Metastore for scalable data handling.
 :iceberg: https://iceberg.apache.org/
 
-WARNING: In NiFi `2.0.0` Iceberg support https://issues.apache.org/jira/browse/NIFI-13938[has been removed].
-
 {iceberg}[Apache Iceberg] is a high-performance format for huge analytic tables.
 Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the same tables, at the same time.
 
 NiFi supports a `PutIceberg` processor to add rows to an existing Iceberg table https://issues.apache.org/jira/browse/NIFI-10442[starting from version 1.19.0].
-As of NiFi version `1.23.1` only `PutIceberg` is supported, you need to create and compact your tables with other tools such as Trino or Spark (both included in the Stackable Data Platform).
+As of NiFi version `2.4.0` only `PutIceberg` is supported, you need to create and compact your tables with other tools such as Trino or Spark (both included in the Stackable Data Platform).
+
+== NiFi 2
+
+In NiFi `2.0.0` Iceberg support https://issues.apache.org/jira/browse/NIFI-13938[has been removed] from upstream NiFi.
+
+We forked the `nifi-iceberg-bundle` and made it available at https://github.com/stackabletech/nifi-iceberg-bundle.
+Starting with SDP 25.7, we have added the necessary bundle to NiFi by default, you don't need to explicitly add Iceberg support to the Stackable NiFi.
+
+Please read on https://github.com/stackabletech/nifi-iceberg-bundle[its documentation] on how to ingest data into Iceberg tables.
+You don't need any special configs on the `NiFiCluster` in case you are using S3 and no Kerberos.
+
+HDFS and Kerberos are also supported, please have a look at the https://github.com/stackabletech/nifi-operator/tree/main/tests/templates/kuttl/iceberg[Iceberg integration test] for that.
+
+== NiFi 1
+
+Starting with `1.19.0`, NiFi supports writing to Iceberg tables.
 
 The following example shows an example NiFi setup using the Iceberg integration.
 

docs/modules/nifi/partials/supported-versions.adoc

@@ -2,7 +2,7 @@
 // This is a separate file, since it is used by both the direct NiFi-Operator documentation, and the overarching
 // Stackable Platform documentation.
 
-* 2.2.0 (experimental) - Please note that you need to upgrade to at least 1.27.x before upgrading to 2.x.x!
+* 2.4.0 (experimental) - Please note that you need to upgrade to at least 1.27.x before upgrading to 2.x.x!
 * 1.28.1
 * 1.27.0 (LTS)