Skip to content

Refactor: Migrate conformance tests to StorageTransport and enhance retry logic#7

Open
thiyaguk09 wants to merge 23 commits intostorage-node-18from
node18/conformance-test
Open

Refactor: Migrate conformance tests to StorageTransport and enhance retry logic#7
thiyaguk09 wants to merge 23 commits intostorage-node-18from
node18/conformance-test

Conversation

@thiyaguk09
Copy link
Owner

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

All 720 test cases have been fixed, and the code has been refactored.
- Replace high-level bucket and file calls with
storageTransport.makeRequest.
- Fix Scenario 1 failures by implementing "create-or-get" logic for
buckets.
- Resolve metageneration mismatch in lock() by dynamically fetching
metadata.
- Normalize header keys to lowercase in transport response processing.
- Increase unit test coverage for shouldRetry logic and error handling.
- Fixed authentication headers/token exchange in the transport layer.
- Reverted to single-shot resumable upload to isolate Scenario 7
  failures while debugging mid-stream offset recovery.
@gemini-code-assist
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the internal conformance tests by migrating them to use the StorageTransport class. This change standardizes how HTTP requests are made and how retries are handled, moving away from direct client methods to a more unified transport layer. The update also introduces more robust retry logic, ensuring that requests are only retried when safe and appropriate, based on HTTP methods and preconditions. This improves the reliability and maintainability of the test suite and the underlying client library.

Highlights

  • Conformance Test Migration: Refactored conformance tests to directly utilize the StorageTransport class, centralizing HTTP request handling and retry logic for consistency.
  • Enhanced Retry Logic: Updated the default retryable error function to intelligently consider HTTP methods and the presence of preconditions (e.g., ifGenerationMatch) when determining if a request should be retried.
  • CI/CD Streamlining: Removed the 'docs' job and its corresponding status check from the CI/CD configuration, simplifying the build process.
  • Test Bench Update: Upgraded the Storage Testbench Docker image version to v0.60.0 for improved testing capabilities.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request refactors Google Cloud Storage conformance tests to directly use StorageTransport.makeRequest for API calls, bypassing higher-level client methods, and updates the testbench Docker image. It also removes the docs CI job and related settings. Review comments highlight a bug in the create function's bucketExists check, where a 404 error was not handled, causing premature exit, and suggest an optimization to reuse the Gaxios instance in the authClient.request mock for efficiency.

Comment on lines +81 to +82
const gaxios = new Gaxios();
return gaxios.request(opts);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Creating a new Gaxios instance for every authClient.request call within this mock can be inefficient, especially if this mock is called frequently. While this is a test utility, reusing a single Gaxios instance might offer a slight performance improvement and better reflect how Gaxios is typically used in production code.

Consider initializing gaxios once outside the request function and reusing it.

authClient.request = async opts => {
  const gaxios = new Gaxios();
  return gaxios.request(opts);
};

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant