diff --git a/config/nav.yml b/config/nav.yml index bf41f54466..fd68c09503 100644 --- a/config/nav.yml +++ b/config/nav.yml @@ -81,6 +81,8 @@ nav: - Install Istio for Knative: install/installing-istio.md # Cert-manager Installation - Install cert-manager: install/installing-cert-manager.md + # Backstage plugin + - Install Knative Backstage plugin: install/installing-backstage-plugins.md # Vendor docs - Using a Knative-based offering: install/knative-offerings.md # Upgrading Knative @@ -274,7 +276,9 @@ nav: - Create a ContainerSource: eventing/custom-event-source/containersource/README.md - ContainerSource Reference: eventing/custom-event-source/containersource/reference.md - Handling delivery failure: eventing/event-delivery.md - - Event registry: eventing/event-registry.md + - Event registry: + - Using the Event registry: eventing/event-registry/README.md + - Knative EventMesh Backstage Plugin: eventing/event-registry/eventmesh-backstage-plugin.md - Sugar controller: eventing/sugar/README.md - Debugging: eventing/troubleshooting/README.md - Administrator topics: diff --git a/docs/eventing/event-registry.md b/docs/eventing/event-registry/README.md similarity index 100% rename from docs/eventing/event-registry.md rename to docs/eventing/event-registry/README.md diff --git a/docs/eventing/event-registry/eventmesh-backstage-plugin.md b/docs/eventing/event-registry/eventmesh-backstage-plugin.md new file mode 100644 index 0000000000..4249802753 --- /dev/null +++ b/docs/eventing/event-registry/eventmesh-backstage-plugin.md @@ -0,0 +1,99 @@ +# Knative Event Mesh Backstage Plugin + +!!! info + For installation instructions, see the [plugin installation documentation](../../../install/installing-backstage-plugins). + +The Knative Event Mesh plugin is a Backstage plugin that allows you to view and manage Knative Eventing resources. The +plugin talks to a special backend that runs in the Kubernetes cluster and communicates with the Kubernetes API server. + +[Backstage](https://backstage.io/){:target="_blank"} is a platform for building developer portals. +It provides a unified way to manage and visualize the different resources that developers use in their daily work. + +While Backstage is not designed as a next generation Kubernetes Dashboard, it can visualize and partially manage Knative +resources. These resources would be read-only and focused on what's relevant to developers. + +![Event Mesh plugin](./images/event-mesh-plugin-components-view.png) + +![Event Mesh plugin](./images/event-mesh-plugin-apis-view.png) + +!!! info + A demo setup for this plugin is available at . + +## How it works + +The plugins are the frontend part of the Backstage instance. They are responsible for rendering the UI and communicating +with the backend. The backend is responsible for talking to the Kubernetes API server and providing the necessary +information to the frontend. + +This plugin leverages Backstage's [entity provider](https://backstage.io/docs/features/software-catalog/external-integrations/#custom-entity-providers){:target="_blank"} and [entity processor](https://backstage.io/docs/features/software-catalog/external-integrations/#custom-processors){:target="_blank"} +concepts. The entity provider is responsible for fetching the resources from the backend and the entity processor is +responsible for processing the resources and making them available to the frontend. +The provider is the part that talks to the custom backend. + +The plugin fetches Knative EventType, Broker, and Trigger resources from the backend. Instead of directly fetching the +raw Kubernetes resources, it gathers a more user-friendly representation of the data. This means certain fields are +excluded, some are combined for clarity, and others are transformed to be easier to understand within this interface. + +## What is shown + +Knative Triggers can be pointing at any [Addressable](https://knative.dev/docs/concepts/duck-typing/#addressable){:target="_blank"} or at even a URL directly. Since Backstage +side won't know how to handle these, the plugin will only show the trigger's subscriber if it is already registered in +Backstage. For this relation, we use the [`backstage.io/kubernetes-id`](https://backstage.io/docs/features/kubernetes/configuration#surfacing-your-kubernetes-components-as-part-of-an-entity){:target="_blank"} +annotation in the Backstage entity and in the Kubernetes resource. + +How the matching works is documented with diagrams in the [Event Mesh plugin README file](https://github.com/knative-extensions/backstage-plugins?tab=readme-ov-file#event-mesh-plugin-1){:target="_blank"}. + +What's really cool about this integration is that Backstage's ability to show graphs. For example, you can see +the relationships between EventTypes, Brokers, and Trigger subscribers in a graph. + +![](./images/event-mesh-plugin-graph.png) +*Graphs in Backstage* + +## Security + +![](./images/knative-backstage-security.png) +*Backstage Security* +[//]: # (https://drive.google.com/file/d/1qMu0yd-zGYcveUM_tLigw1yZ_0jksX9i/view?usp=drive_link) + +Similar to other Backstage plugins, we wanted the plugin administrator to configure the plugin by setting up the +necessary things like the backend URL and the token. It is a similar approach with the +[Backstage Kubernetes plugin](https://backstage.io/docs/features/kubernetes/configuration#configuring-kubernetes-clusters), where the user needs to provide the URL and the token. + +The token is stored in Backstage configuration and is passed to the backend with each request. +The backend uses this token to authenticate to the Kubernetes API server. +The token is a service account token that has the necessary permissions to list the Knative Eventing +resources in the cluster. + +```yaml +... +catalog: + providers: + knativeEventMesh: + dev: + token: '${KNATIVE_EVENT_MESH_TOKEN}' + baseUrl: "http://eventmesh-backend.knative-eventing.svc:8080" + schedule: # optional; same options as in TaskScheduleDefinition + # supports cron, ISO duration, "human duration" as used in code + frequency: { minutes: 1 } + # supports ISO duration, "human duration" as used in code + timeout: { minutes: 1 } +``` + +The `token` is taken from the `KNATIVE_EVENT_MESH_TOKEN` environment variable. Backstage supports environment variables +in the configuration files, so you can set the token as an environment variable before starting the Backstage instance. +Actually, Backstage has other mechanisms, including configuration files, file includes and others. +You can check the [Backstage documentation](https://backstage.io/docs/conf/writing/){:target="_blank"} for more information. + +How to create the `ServiceAccount`, `ClusterRole`, `ClusterRoleBinding`, `Secret` and the token for that `Secret` is documented in +the [plugin installation documentation](../../../install/installing-backstage-plugins). + +## Usage + +The plugin will show all `Broker` and `EventType` resources in the cluster. `Broker`s will be shown as Backstage +[`Component`s](https://backstage.io/docs/features/software-catalog/system-model#component){:target="_blank"} +and `EventType`s will be shown as Backstage [`API`s](https://backstage.io/docs/features/software-catalog/system-model#api){:target="_blank"}. + +The subscribers of the `Trigger`s will be shown as Backstage `Component`s. However, they will be shown if: + +- They are registered in Backstage +- They have the [`backstage.io/kubernetes-id`](https://backstage.io/docs/features/kubernetes/configuration#surfacing-your-kubernetes-components-as-part-of-an-entity){:target="_blank"} annotation set to the Kubernetes resource's name diff --git a/docs/eventing/event-registry/images/event-mesh-plugin-apis-view.png b/docs/eventing/event-registry/images/event-mesh-plugin-apis-view.png new file mode 100644 index 0000000000..cbcc111dfc Binary files /dev/null and b/docs/eventing/event-registry/images/event-mesh-plugin-apis-view.png differ diff --git a/docs/eventing/event-registry/images/event-mesh-plugin-components-view.png b/docs/eventing/event-registry/images/event-mesh-plugin-components-view.png new file mode 100644 index 0000000000..664a48d363 Binary files /dev/null and b/docs/eventing/event-registry/images/event-mesh-plugin-components-view.png differ diff --git a/docs/eventing/event-registry/images/event-mesh-plugin-graph.png b/docs/eventing/event-registry/images/event-mesh-plugin-graph.png new file mode 100644 index 0000000000..64bec60ddc Binary files /dev/null and b/docs/eventing/event-registry/images/event-mesh-plugin-graph.png differ diff --git a/docs/eventing/event-registry/images/knative-backstage-security.png b/docs/eventing/event-registry/images/knative-backstage-security.png new file mode 100644 index 0000000000..8ea724a3bf Binary files /dev/null and b/docs/eventing/event-registry/images/knative-backstage-security.png differ diff --git a/docs/install/installing-backstage-plugins.md b/docs/install/installing-backstage-plugins.md new file mode 100644 index 0000000000..fad199ed25 --- /dev/null +++ b/docs/install/installing-backstage-plugins.md @@ -0,0 +1,270 @@ +# Installing Knative Backstage Plugins + +Knative community is planning to provide a set of Backstage plugins for Knative and their respective backends. +Currently there is one plugin available, the Event Mesh plugin. + +## Event Mesh plugin + +The Event Mesh plugin is a Backstage plugin that allows you to view and manage Knative Eventing resources. + +The Backstage plugin talks to a special backend that runs in the Kubernetes cluster and communicates with the Kubernetes +API server. + +A demo setup for this plugin is available at . + +The plugin has 2 distributions: static and dynamic. In this document, we will focus on the static distribution. +For the dynamic distribution, please see the +[Dynamic Plugin README file](https://github.com/knative-extensions/backstage-plugins/blob/main/backstage/plugins/knative-event-mesh-backend/README-dynamic.md) +in the plugin repository. + +### Installation + +The plugin needs to be installed in the Backstage instance and the backend it talks to needs to be installed in the +Kubernetes cluster. + +#### Plugin backend controller installation + +```shell +VERSION="latest" # or a specific version like knative-v1.15.0 +kubectl apply -f https://github.com/knative-extensions/backstage-plugins/releases/${VERSION}/download/eventmesh.yaml +``` + +This will install the backend controller in the Kubernetes cluster. The backend's responsibility is to talk to +the Kubernetes API server and provide the necessary information to the plugin. + +#### The Backstage plugin installation + +In your Backstage directory, run the following command to install the plugin: + +```bash +VERSION="latest" # or a specific version like 1.15.0 from https://www.npmjs.com/package/@knative-extensions/plugin-knative-event-mesh-backend +yarn workspace backend add @knative-extensions/plugin-knative-event-mesh-backend@${VERSION} +``` + +Backstage has a legacy backend system that is being replaced with a new system. If you are using the legacy backend +system, you can follow the instructions below to install the plugin. + +To learn more about the new and the legacy backend systems, see the +[Backstage documentation](https://backstage.io/docs/backend-system/building-backends/migrating/). + +!!! info + We are aware there is a `Backend` term used in both the Kubernetes controller and the Backstage backend system. + Backstage backend system is different from the Kubernetes controller we've installed before. + The controller is a Kubernetes controller that runs in the Kubernetes cluster and talks to the Kubernetes API server. + Backstage backend system is a framework to run backend plugins that talk to data providers, such as the Kubernetes controller mentioned above. + +#### Enabling the plugin on the new Backstage backend system + +To install on the new backend system, add the following into the `packages/backend/index.ts` file: + +```ts +import { createBackend } from '@backstage/backend-defaults'; + +const backend = createBackend(); + +// Other plugins/modules + +backend.add(import('@knative-extensions/plugin-knative-event-mesh-backend/alpha')); +``` + +!!! warning + If you have made any changes to the schedule in the `app-config.yaml` file, then restart to apply the changes. + +#### Enabling the plugin on the legacy Backstage backend system + +Configure the scheduler for the entity provider and enable the processor. Add the following code +to `packages/backend/src/plugins/catalog.ts` file: + +```ts +import {CatalogClient} from "@backstage/catalog-client"; +import { + KnativeEventMeshProcessor, + KnativeEventMeshProvider +} from '@knative-extensions/plugin-knative-event-mesh-backend'; + +export default async function createPlugin( + env:PluginEnvironment, +):Promise { + const builder = await CatalogBuilder.create(env); + + /* ... other processors and/or providers ... */ + + // ADD THESE + builder.addEntityProvider( + KnativeEventMeshProvider.fromConfig(env.config, { + logger: env.logger, + scheduler: env.scheduler, + }), + ); + const catalogApi = new CatalogClient({ + discoveryApi: env.discovery, + }); + const knativeEventMeshProcessor = new KnativeEventMeshProcessor(catalogApi, env.logger); + builder.addProcessor(knativeEventMeshProcessor); + + /* ... other processors and/or providers ... */ + + const {processingEngine, router} = await builder.build(); + await processingEngine.start(); + return router; +} +``` + +### Configuration + +!!! info + **NOTE**: The backend needs to be accessible from the Backstage instance. If you are running the backend without + exposing it, you can use `kubectl port-forward` to forward the port of the backend service to your local machine + for testing purposes. + ```bash + kubectl port-forward -n knative-eventing svc/eventmesh-backend 8080:8080 + ``` + +The plugin needs to be configured to talk to the backend. It can be configured in the `app-config.yaml` file of the +Backstage instance and allows configuration of one or multiple providers. + +Use a `knativeEventMesh` marker to start configuring the `app-config.yaml` file of Backstage: + +```yaml +catalog: + providers: + knativeEventMesh: + dev: + token: '${KNATIVE_EVENT_MESH_TOKEN}' # SA token to authenticate to the backend + baseUrl: '${KNATIVE_EVENT_MESH_BACKEND}' # URL of the backend installed in the cluster + schedule: # optional; same options as in TaskScheduleDefinition + # supports cron, ISO duration, "human duration" as used in code + frequency: { minutes: 1 } + # supports ISO duration, "human duration" as used in code + timeout: { minutes: 1 } +``` + +You can either manually change the placeholders in the `app-config.yaml` file or use environment variables to set the +values. The environment variables can be set as following before starting the Backstage instance: + +```bash +export KNATIVE_EVENT_MESH_TOKEN= +export KNATIVE_EVENT_MESH_BACKEND= +``` + +The value of `KNATIVE_EVENT_MESH_BACKEND` should be the URL of the backend service. If you are running the backend +service in the same cluster as the Backstage instance, you can use the service name as the URL such as +`http://eventmesh-backend.knative-eventing.svc.cluster.local`. +If the Backstage instance is not running in the same cluster, you can use the external URL of the backend service. +Or, if you are running the backend without exposing it for testing purposes, you can use `kubectl port-forward` as +mentioned above. + +The value of `KNATIVE_EVENT_MESH_TOKEN` should be a service account token that has the necessary permissions to list +the Knative Eventing resources in the cluster. The backend will use this token to authenticate to the Kubernetes API +server. This is required for security reasons as otherwise (if the backend is running with a SA token directly) the +backend would have full access to the cluster will be returning all resources to anyone who can access the backend. + +The token will require the following permissions to work properly: + +- `get`, `list` and `watch` permissions for `eventing.knative.dev/brokers`, `eventing.knative.dev/eventtypes` and + `eventing.knative.dev/triggers` resources +- `get` permission for all resources to fetch subscribers for triggers + +You can create a ClusterRole with the necessary permissions and bind it to the service account token. + +An example configuration is as follows: + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: my-eventmesh-backend-service-account + namespace: default +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: my-eventmesh-backend-cluster-role +rules: + # permissions for eventtypes, brokers and triggers + - apiGroups: + - "eventing.knative.dev" + resources: + - brokers + - eventtypes + - triggers + verbs: + - get + - list + - watch + # permissions to get subscribers for triggers + # as subscribers can be any resource, we need to give access to all resources + # we fetch subscribers one by one, we only need `get` verb + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: my-eventmesh-backend-cluster-role-binding +subjects: + - kind: ServiceAccount + name: my-eventmesh-backend-service-account + namespace: default +roleRef: + kind: ClusterRole + name: my-eventmesh-backend-cluster-role + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: v1 +kind: Secret +metadata: + name: my-eventmesh-backend-secret + namespace: default + annotations: + kubernetes.io/service-account.name: my-eventmesh-backend-service-account +type: kubernetes.io/service-account-token +``` + +To get the token, you can run the following command: + +```bash +kubectl get secret my-eventmesh-backend-secret -o jsonpath='{.data.token}' | base64 --decode +``` + +Run a quick check to see if the token works with the Kubernetes API server: + +```bash +export KUBE_API_SERVER_URL=$(kubectl config view --minify --output jsonpath="{.clusters[*].cluster.server}") # e.g. "https://192.168.2.151:16443" +export KUBE_SA_TOKEN=$(kubectl get secret my-eventmesh-backend-secret -o jsonpath='{.data.token}' | base64 --decode) +curl -k -H "Authorization: Bearer $KUBE_SA_TOKEN" -X GET "${KUBE_API_SERVER_URL}/apis/eventing.knative.dev/v1/namespaces/default/brokers" +# Should see the brokers, or nothing if there are no brokers +# But, should not see an error +``` + +Run a second quick check to see if the token works with the Backstage backend: + +```bash +export KNATIVE_EVENT_MESH_BACKEND=http://localhost:8080 # or the URL of the backend +export KUBE_SA_TOKEN=$(kubectl get secret my-eventmesh-backend-secret -o jsonpath='{.data.token}' | base64 --decode) +curl -k -H "Authorization: Bearer $KUBE_SA_TOKEN" -X GET "${KNATIVE_EVENT_MESH_BACKEND}" +# Should see the response from the backend such as +# { +# "brokers" : [...], +# "eventTypes" : [...] +#} +``` + +If these quick checks work, you can use the token in the `app-config.yaml` file as the value +of `KNATIVE_EVENT_MESH_TOKEN`. + +### Troubleshooting + +When you start your Backstage application, you can see some log lines as follows: + +```text +[1] 2024-01-04T09:38:08.707Z knative-event-mesh-backend info Found 1 knative event mesh provider configs with ids: dev type=plugin +``` + +### Usage + +See the [plugin documentation](../../eventing/event-registry/eventmesh-backstage-plugin/) for more information about using the plugin.