Skip to content

Native SSH Tunnel support for database connections #6573#6776

Merged
hansva merged 2 commits intoapache:mainfrom
pyjams:main
Mar 16, 2026
Merged

Native SSH Tunnel support for database connections #6573#6776
hansva merged 2 commits intoapache:mainfrom
pyjams:main

Conversation

@pyjams
Copy link
Contributor

@pyjams pyjams commented Mar 13, 2026

Native SSH Tunnel support for database connections

Addresses #6573

Problem

Currently, Apache Hop has no built-in way to connect to databases that sit inside private networks behind an SSH bastion host. Users are forced to set up external SSH tunnels manually (e.g. via ssh -L) before running pipelines, which complicates deployment and is error-prone in automated/containerised environments.

Solution

This PR adds native SSH tunnel support directly in the database connection metadata, so that Hop can transparently open an SSH tunnel before establishing the JDBC connection — no external tooling required.

What changed

  • SshTunnelManager (new class): manages JSch SSH sessions with local port forwarding. Supports three authentication methods: password, keyboard-interactive, and private key (with optional passphrase). Includes a 30-second keepalive interval to prevent VPN/firewall idle-connection drops.
  • IDatabase / BaseDatabaseMeta: added SSH tunnel configuration fields (sshEnabled, sshHost, sshPort, sshUsername, sshPassword, sshPrivateKey, sshPassphrase) persisted via @HopMetadataProperty.
  • DatabaseMeta: delegate getters/setters for the new SSH fields.
  • Database: opens the SSH tunnel before the JDBC connect and rewrites the JDBC URL to point to localhost:<localPort>. The tunnel is closed in closeConnectionOnly() (not disconnect()) to avoid premature teardown when connections are shared across a pipeline connection group.
  • DatabaseMetaEditor (UI): new "SSH Tunnel" tab in the database connection dialog, with fields that enable/disable dynamically based on the selected authentication method.
  • i18n: English and Italian labels.

Testing

  • 5 unit tests for SshTunnelManager (session creation, port forwarding, authentication modes, error handling)
  • 5 unit tests for Database SSH tunnel integration (tunnel open/close lifecycle, URL rewriting, grouped connections)
  • Manual testing verified with MySQL over SSH tunnel

Checklist

Dependencies

Added com.jcraft:jsch:0.1.55 to core/pom.xml (widely used, Apache-compatible BSD license).

Note: pre-existing LDAP build failure

The hop-transform-ldap module fails to compile on current main due to commit 859d0dbc (PR #6665 — LDAP XML cleanup) which changed several methods in LdapOutputMeta and LdapInputMeta to private that are still called by external converter classes (LdapOutputOperationTypeConverter, LdapOutputDerefAliasesTypeConverter, LdapOutputReferralTypeConverter, LdapInputSearchScopeConverter). This is unrelated to the changes in this PR.

@hansva
Copy link
Contributor

hansva commented Mar 13, 2026

Thanks for this @pyjams!

@hansva
Copy link
Contributor

hansva commented Mar 13, 2026

We removed the vintage Junit engine and migrated all our tests to Junit 5+. I can take a look at that if you want

@pyjams
Copy link
Contributor Author

pyjams commented Mar 13, 2026

We removed the vintage Junit engine and migrated all our tests to Junit 5+. I can take a look at that if you want

Thank you, I’ve just updated the files. There shouldn’t be any errors now.

Giacomo and others added 2 commits March 16, 2026 10:56
…ers to connect

  to databases in private networks through an SSH bastion host
- Supports password, keyboard-interactive, and private key authentication
- New "SSH Tunnel" tab in the database connection editor with enable/disable logic
- SSH keepalive (30s interval) to prevent VPN/firewall from dropping idle connections
- Tunnel lifecycle properly managed even for pipeline connection groups

- **SshTunnelManager** (new): manages JSch sessions with local port forwarding
- **IDatabase/BaseDatabaseMeta**: SSH tunnel fields persisted via `@HopMetadataProperty`
- **Database.java**: opens tunnel before JDBC connect, closes in `closeConnectionOnly()`
  (not `disconnect()`) to prevent tunnel leaks with grouped connections
- **DatabaseMetaEditor**: new SSH Tunnel tab with field enable/disable based on config
- i18n: English and Italian labels

- [x] Unit tests for SshTunnelManager (5 tests)
- [x] Unit tests for Database SSH tunnel integration (5 tests)
- [x] All 10 tests pass
- [x] Manual testing with MySQL over SSH tunnel (verified working)

UPGRADE JUNIT 5

revert ldap changes
@hansva
Copy link
Contributor

hansva commented Mar 16, 2026

I removed the LDAP changes and added integration tests.
Looking very good, thanks for the work @pyjams

@hansva hansva merged commit bba2fa8 into apache:main Mar 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants