Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: 'recursive'

- name: Setup .NET
uses: actions/setup-dotnet@v3
Expand All @@ -26,4 +28,14 @@ jobs:
run: dotnet build --configuration Release --no-restore

- name: Test
run: dotnet test --configuration Release --no-build --verbosity normal
run: dotnet test --configuration Release --no-build --verbosity normal --logger "junit;LogFileName=unit.junit"

- name: View unit test results
uses: dorny/test-reporter@v1
if: success() || failure()
with:
name: Unit Test Results
path: '**/unit.junit'
reporter: java-junit
fail-on-error: true
only-summary: false
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "IO.Ably.DeltaCodec.Test/vcdiff-tests"]
path = IO.Ably.DeltaCodec.Test/vcdiff-tests
url = https://github.com/ably/vcdiff-tests.git
13 changes: 13 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## Contributing

1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Ensure you have added suitable tests and the test suite is passing (`dotnet test`)
5. Push to the branch (`git push origin my-new-feature`)
6. Create a new Pull Request

## Release Process

- Make sure the tests are passing in ci for the branch you're building
- Makr sure to get atleast one approval for PR and then merge to main.
12 changes: 9 additions & 3 deletions IO.Ably.DeltaCodec.Test/IO.Ably.DeltaCodec.Test.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,14 @@
</ProjectReference>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />
<PackageReference Include="NUnit" Version="3.12.0" />
<PackageReference Include="NUnit3TestAdapter" Version="3.13.0" />
<PackageReference Include="JunitXml.TestLogger" Version="4.1.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
<PackageReference Include="NUnit" Version="4.2.2" />
<PackageReference Include="NUnit3TestAdapter" Version="4.6.0" />
<PackageReference Include="System.Text.Json" Version="8.0.5" />
</ItemGroup>
<ItemGroup>
<!-- Copy all vcdiff-tests files to output directory -->
<None Include="vcdiff-tests\**\*" CopyToOutputDirectory="PreserveNewest" LinkBase="vcdiff-tests" />
</ItemGroup>
</Project>
101 changes: 101 additions & 0 deletions IO.Ably.DeltaCodec.Test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Delta Codec Test Suite

This directory contains comprehensive tests for the VCDIFF decoder implementation.

## Test Structure

### 1. VcdiffDecoderFixture.cs
Legacy tests using the original xdelta test data located in `TestData/xdelta/`. These tests validate basic decoder functionality with 4 test cases.

### 2. VcdiffTestSuiteFixture.cs (NEW)
Comprehensive test suite using the [vcdiff-tests](https://github.com/ably/vcdiff-tests) submodule. This provides **85 standardized tests** across multiple categories:

#### Test Categories

**Targeted Positive Tests** (29 tests)
- Basic operations: empty files, content changes, duplications, unchanged files
- Varint boundary tests: All varint encoding boundaries (0, 127, 128, 16383, 16384, 2097151, 2097152) for ADD, COPY, and RUN instructions
- Codetable tests: Complete coverage of all 256 VCDIFF codetable entries
- Address cache tests: Near cache, same cache, different addressing modes
- Checksum tests: VCD_ADLER32 validation

**Targeted Negative Tests** (33 tests)
- Invalid headers: Wrong magic bytes, unsupported versions
- Malformed windows: Invalid indicators, impossible sizes
- Bad instructions: Invalid instruction codes, out-of-bounds references
- Checksum failures: Incorrect Adler32 checksums
- Format violations: Truncated files, invalid varints

**General Positive Tests** (20 tests)
- Binary files: 64 bytes, 1KB, 64KB with append/delete/insert/modify operations
- JSON files: 1KB, 64KB with structure-preserving modifications

**Fuzz Tests** (optional)
- Corrupted input validation (ensures decoder doesn't crash)

## Test Format

Each test case in the vcdiff-tests submodule consists of:
- `source` - Original file (may be empty)
- `target` - Expected result after applying delta
- `delta.vcdiff` - VCDIFF delta file
- `metadata.json` - Test description and expected behavior

## Running Tests

Run all tests:
```bash
dotnet test
```

Run only the new comprehensive test suite:
```bash
dotnet test --filter "FullyQualifiedName~VcdiffTestSuiteFixture"
```

Run specific test categories:
```bash
# Targeted positive tests
dotnet test --filter "FullyQualifiedName~VcdiffTestSuiteFixture.TestTargetedPositive"

# Targeted negative tests
dotnet test --filter "FullyQualifiedName~VcdiffTestSuiteFixture.TestTargetedNegative"

# General positive tests
dotnet test --filter "FullyQualifiedName~VcdiffTestSuiteFixture.TestGeneralPositive"
```

## Test Results

Current test status:
- ✅ **All tests passing**

All 85 tests from the comprehensive vcdiff-tests suite are passing successfully, along with the 4 legacy xdelta tests.

## Updating Test Data

The vcdiff-tests submodule can be updated to get the latest test cases:

```bash
cd IO.Ably.DeltaCodec.Test/vcdiff-tests
git pull origin main
cd ../..
git add IO.Ably.DeltaCodec.Test/vcdiff-tests
git commit -m "Update vcdiff-tests submodule"
```

## Implementation Details

The test implementation in `VcdiffTestSuiteFixture.cs` mirrors the Go implementation from [vcdiff-go](https://github.com/ably/vcdiff-go), providing:

1. **Automatic test discovery**: Recursively finds all test cases in each category
2. **Metadata parsing**: Reads and validates test metadata from JSON files
3. **Dynamic test generation**: Creates NUnit test cases for each discovered test
4. **Comprehensive validation**: Compares decoded output byte-by-byte with expected targets
5. **Error handling validation**: Ensures negative tests fail as expected

## References

- [RFC 3284: VCDIFF Format Specification](https://tools.ietf.org/html/rfc3284)
- [vcdiff-tests Repository](https://github.com/ably/vcdiff-tests)
- [vcdiff-go Implementation](https://github.com/ably/vcdiff-go)
2 changes: 1 addition & 1 deletion IO.Ably.DeltaCodec.Test/VcdiffDecoderFixture.cs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ public void DecodeXdeltaPatch(string testCasePath)
}

byte[] target = File.ReadAllBytes(targetPath);
Assert.IsTrue(target.SequenceEqual(decoded), "Delta application result does not match the expected target file.");
Assert.That(target.SequenceEqual(decoded), Is.True, "Delta application result does not match the expected target file.");
}
}
}
Loading
Loading