Skip to content

docs(pulsar): add CloudEvents schema wrapping and rawSchema option for Avro and JSON schemas#5084

Open
javier-aliaga wants to merge 3 commits intodapr:v1.17from
javier-aliaga:pulsar-avro-cloudevents
Open

docs(pulsar): add CloudEvents schema wrapping and rawSchema option for Avro and JSON schemas#5084
javier-aliaga wants to merge 3 commits intodapr:v1.17from
javier-aliaga:pulsar-avro-cloudevents

Conversation

@javier-aliaga
Copy link
Copy Markdown
Contributor

@javier-aliaga javier-aliaga commented Mar 24, 2026

Summary

  • Document CloudEvents envelope wrapping behavior for both Avro and JSON schemas registered with the Pulsar Schema Registry
  • Add the <topic-name>.rawschema metadata field, which applies to both schema types, for topics that need to skip CE wrapping and register the inner schema directly
  • Include a behavior matrix table, configuration examples (Avro + JSON), and misconfiguration warnings

Details

When CloudEvents wrapping is enabled (the default), Dapr wraps user-provided schemas inside a CloudEvents envelope before registering them with Pulsar. The new .rawschema metadata flag lets operators opt out of
this wrapping on a per-topic basis for both .avroschema and .jsonschema topics.

Ref: dapr/components-contrib#4302
Ref: dapr/components-contrib#4305

Thank you for helping make the Dapr documentation better!

Please follow this checklist before submitting:

  • Commits are signed with Developer Certificate of Origin (DCO - learn more)
  • Read the contribution guide
  • Commands include options for Linux, MacOS, and Windows within tabpane
  • New file and folder names are globally unique
  • Page references use shortcodes instead of markdown or URL links
  • Images use HTML style and have alternative text
  • Places where multiple code/command options are given have tabpane

In addition, please fill out the following to help reviewers understand this pull request:

Description

Issue reference

@javier-aliaga javier-aliaga requested review from a team as code owners March 24, 2026 15:22
Document the new behavior where Dapr wraps user-provided Avro schemas
inside a CloudEvents envelope before registering with the Pulsar Schema
Registry. Add the new `<topic-name>.rawPayload` metadata field for
topics that need to skip CE wrapping and register the inner schema
directly.

Ref: dapr/components-contrib#4302
Signed-off-by: Javier Aliaga <javier@diagrid.io>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Pulsar pub/sub component documentation to explain Avro schema registration behavior when publishing CloudEvents, and introduces a component-level option to register the “inner” Avro schema for raw-payload-only topics.

Changes:

  • Document that <topic-name>.avroschema is wrapped in a CloudEvents Avro envelope schema when CloudEvents wrapping is enabled (default).
  • Add <topic-name>.rawSchema metadata to register the inner Avro schema directly (for raw-payload-only topics).
  • Add a behavior matrix and example configuration for splitting CE-wrapped vs raw-payload topics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Document the new behavior where Dapr wraps user-provided Avro schemas
inside a CloudEvents envelope before registering with the Pulsar Schema
Registry. Add the new `<topic-name>.rawPayload` metadata field for
topics that need to skip CE wrapping and register the inner schema
directly.

Ref: dapr/components-contrib#4302
Signed-off-by: Javier Aliaga <javier@diagrid.io>
@javier-aliaga javier-aliaga force-pushed the pulsar-avro-cloudevents branch from 7340081 to 290a970 Compare March 25, 2026 08:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…hema

Update documentation to reflect that the rawschema metadata flag works
with both Avro and JSON schema types, not just Avro. Add JSON schema
CloudEvents wrapping description and JSON schema example in the YAML
config block.

Signed-off-by: Javier Aliaga <javier@diagrid.io>
@javier-aliaga javier-aliaga force-pushed the pulsar-avro-cloudevents branch from 7e3321c to b06d0ca Compare March 28, 2026 10:29
@javier-aliaga javier-aliaga changed the title docs(pulsar): add CloudEvents Avro schema wrapping and rawPayload option docs(pulsar): add CloudEvents schema wrapping and rawSchema option for Avro and JSON schemas Mar 28, 2026
@javier-aliaga javier-aliaga changed the base branch from v1.17 to v1.16 March 28, 2026 10:32
@javier-aliaga javier-aliaga changed the base branch from v1.16 to v1.17 March 28, 2026 10:33
@javier-aliaga javier-aliaga requested a review from Copilot March 28, 2026 10:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +87 to +88
| <topic-name>.jsonschema | N | Enforces JSON schema validation for the configured topic. When CloudEvents wrapping is enabled (the default), the schema registered with the Pulsar Schema Registry is a CloudEvents envelope JSON schema containing the provided schema as the `data` field. |
| <topic-name>.avroschema | N | Enforces Avro schema validation for the configured topic. When CloudEvents wrapping is enabled (the default), the schema registered with the Pulsar Schema Registry is a CloudEvents envelope Avro schema containing the provided schema as the `data` field. |
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec metadata table has 4 columns (Field/Required/Details/Example), but the new <topic-name>.jsonschema and <topic-name>.avroschema rows only provide 3 cells (missing the Example column). This breaks the Markdown table rendering/alignment; add an Example value (and trailing |) for both rows to match the header.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants