Skip to content

Commit b7c49c1

Browse files
aboitreaudwarrierrurseberry
authored
Data Jobs Monitoring: set env on Composer (#33244)
* set env on composer * Update content/en/data_jobs/airflow.md Co-authored-by: Ryan Warrier <[email protected]> * remove py package + flag default with Dataplex * vale * Update content/en/data_jobs/airflow.md Co-authored-by: Ursula Chen <[email protected]> * PR suggestion tradeoff * Add a missing word --------- Co-authored-by: Ryan Warrier <[email protected]> Co-authored-by: Ursula Chen <[email protected]>
1 parent 7188992 commit b7c49c1

File tree

1 file changed

+9
-8
lines changed

1 file changed

+9
-8
lines changed

content/en/data_jobs/airflow.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -306,7 +306,7 @@ Data Jobs Monitoring for Airflow is not yet compatible with <a href=https://clou
306306
To get started, follow the instructions below.
307307
308308
309-
1. In the Advanced Configuration tab, under **Airflow configuration override**, click **Add Airflow configuration override** and configure these settings:
309+
In the Advanced Configuration tab, under **Airflow configuration override**, click **Add Airflow configuration override** and configure these settings:
310310
311311
- In Section 1, enter `openlineage`.
312312
- In Key 1, enter `disabled`.
@@ -330,20 +330,21 @@ To get started, follow the instructions below.
330330
* Replace `<DD_DATA_OBSERVABILITY_INTAKE>` fully with `https://data-obs-intake.`{{< region-param key="dd_site" code="true" >}}.
331331
* Replace `<DD_API_KEY>` fully with your valid [Datadog API key][5].
332332
333+
**Optional:** Configure the OpenLineage namespace to set the `env` tag value in Datadog:
334+
- In Section 3, enter `openlineage`.
335+
- In Key 3, enter `namespace`.
336+
- In Value 3, enter your Composer environment value (for example, prod, dev, staging, or test).
337+
338+
**Note:** If [Dataplex][6] is enabled, the namespace is already set by default to the Composer environment name. Manually setting the OpenLineage namespace here optionally allows you to override this default value.
333339
334-
Check official [Airflow][4] and [Composer][3] documentation pages for other supported configurations of the `openlineage` provider in Google Cloud Composer.
335-
336-
2. After starting the Composer environment, install the `openlineage` provider by adding the following package in the Pypi packages tab of your environment page:
337-
```text
338-
apache-airflow-providers-openlineage
339-
```
340-
340+
Check official [Airflow][4] and [Composer][3] documentation pages for other supported configurations of the `openlineage` provider in Google Cloud Composer.
341341
342342
[1]: https://cloud.google.com/composer/docs/composer-versioning-overview
343343
[2]: https://airflow.apache.org/docs/apache-airflow-providers-openlineage/stable/index.html
344344
[3]: https://cloud.google.com/composer/docs/airflow-configurations
345345
[4]: https://airflow.apache.org/docs/apache-airflow-providers-openlineage/stable/configurations-ref.html#configuration-openlineage
346346
[5]: https://docs.datadoghq.com/account_management/api-app-keys/#api-keys
347+
[6]: https://cloud.google.com/composer/docs/composer-2/lineage-integration
347348
[7]: https://app.datadoghq.com/data-jobs/
348349
349350
### Validation

0 commit comments

Comments
 (0)