Debezium PostgreSQL source connector for pulsar 5.0.0

[PudgeJ]

Hi,

I'm using Apache Pulsar 5.0.0 and need the Debezium PostgreSQL source connector (.nar file).

I've checked the official Pulsar documentation and download page, but I couldn't find a download link for the Debezium source connector compatible with Pulsar 5.0.0.

Could someone please point me to the official download location for the connector NAR file, or explain how to build it from source?

I specifically need the PostgreSQL Debezium source connector compatible with Pulsar 5.0.0.

Thank you!

[abozaralizadeh]

There is no 5.0.0 NAR. Since 5.0 the connectors aren't released with Pulsar anymore, they moved to their own repo, apache/pulsar-connectors (PIP-465), and that repo has no release yet. That's why the 5.0.0 download folder only has the bin, src, offloaders and shell tarballs. (The 5.0.x docs page still says pulsar-io-debezium-postgres-5.0.0.nar, but that file doesn't exist.)

What works for now is the 4.2.5 NAR (PIP-465 says older connector NARs keep working):

https://downloads.apache.org/pulsar/pulsar-4.2.5/connectors/pulsar-io-debezium-postgres-4.2.5.nar

One catch if you use the official apachepulsar/pulsar:5.0.0 Docker image. I tried it and the source dies with:

java.lang.NoClassDefFoundError: javax/lang/model/SourceVersion
	at io.debezium.config.Field.isClassName(Field.java:1259)

The 5.0.0 image ships a slimmed JVM and java.compiler was left out on purpose (Dockerfile L71-73), but Debezium needs it. With a full JVM it works. I put the full Corretto 25 JDK back into the image:

FROM amazoncorretto:25-alpine AS jdk

FROM apachepulsar/pulsar:5.0.0
USER root
RUN rm -rf /opt/jvm
COPY --from=jdk /usr/lib/jvm/default-jvm /opt/jvm
USER 10000

With that image, Postgres 16 (wal_level=logical), the 4.2.5 NAR and the Postgres config from the docs, pulsar-admin source localrun gave me both the snapshot row and a new insert on public/default/dbserver1.public.users. If you run the tarball on a normal full JDK you shouldn't hit this at all.

If you'd rather build the NAR yourself, the connectors repo uses Gradle now (JDK 17+):

git clone https://github.com/apache/pulsar-connectors
cd pulsar-connectors
./gradlew :debezium:pulsar-io-debezium-postgres:assemble

That gave me debezium/postgres/build/libs/pulsar-io-debezium-postgres-4.3.0-SNAPSHOT.nar (it's a snapshot, so I tested with the released 4.2.5 one).

The missing java.compiler in the image is probably worth an issue, anyone running Debezium on the 5.0.0 image will hit it. Does that get you going?

[PudgeJ]

I haven't found a solution yet. I want to wait for the official one. Any workaround might cause unpredictable problems. If there is no Debezium source connector in version 5.0.0, perhaps I can only continue to use version 4.2.5.

[abozaralizadeh]

@PudgeJ there's an official fix now. Pulsar 5.0.1 is out (Oct 26) and its image has java.compiler again (#27035, it's in the 5.0.1 release notes).

I ran the stock apachepulsar/pulsar:5.0.1 image, no changes, with the official 4.2.5 NAR and Postgres 16. No NoClassDefFoundError this time, the snapshot finished and the row showed up on dbserver1.public.users.

The connector is still the 4.2.5 one, since pulsar-connectors has no release yet. But 5.0.1 plus the released 4.2.5 NAR means no custom image and no snapshot build.

[abozaralizadeh]

@PudgeJ there's an official fix now. Pulsar 5.0.1 is out (Oct 26) and its image has java.compiler again (#27035, it's in the 5.0.1 release notes).

I ran the stock apachepulsar/pulsar:5.0.1 image, no changes, with the official 4.2.5 NAR and Postgres 16. No NoClassDefFoundError this time, the snapshot finished and the row showed up on dbserver1.public.users.

The connector is still the 4.2.5 one, since pulsar-connectors has no release yet. But 5.0.1 plus the released 4.2.5 NAR means no custom image and no snapshot build.

[PudgeJ]

@abozaralizadeh Thanks, bro! A Pulsar contributor has fixed the java.compiler issue in the Docker image, and he said it works fine with the 4.2.5 .nar file. I haven't tried it yet.

Have you encountered any issues so far when using Pulsar 5.0.1 with the 4.2.5 .nar file?