[Task]: Setup a GCS based delta lake repo for integration testing

[chamikaramj]

What needs to happen?

For context, please see https://s.apache.org/beam-delta-lake-source.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Prism Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[utkarshparekh]

@chamikaramj I can take this. I noticed the ignored test in DeltaIOTest still points at a local test_table_1_gb path. Do you have a preferred bucket/path and table set for the shared GCS repo, and should this complement the Dataflow IT in #38954? Happy to put together a PR either way.

[chamikaramj]

Actually, we might not need this. We can setup repo in GCS easily when running a given test and delete at the end.

[chamikaramj]

We already have Dataflow integration tests that setup a Delta Lake repo in GCS.

https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_Delta_IO_Dataflow.yml

Closing this.