When copying data from drs to gs with drs.copy(), it copies files larger than 100 MB in chunks via _copy_multipart_passthrough. That function performs an md5 check:
if dst_blob.md5 != src_blob.md5:
For GCP, the md5 property is retrieved via:
@property
def md5(self) -> str:
gs_md5 = self._get_native_blob().md5_hash
return base64.b64decode(gs_md5).hex()
However, composite objects do not have the md5_hash property:
Composite objects do not have an MD5 hash metadata field.
Passing None to base64.b64decode results in an error. GCP still provides a crc32c hash.
When copying data from drs to gs with
drs.copy(), it copies files larger than 100 MB in chunks via _copy_multipart_passthrough. That function performs an md5 check:For GCP, the md5 property is retrieved via:
However, composite objects do not have the md5_hash property:
Passing
Nonetobase64.b64decoderesults in an error. GCP still provides a crc32c hash.