Repository navigation
S3 File Writer: Custom HTTP Headers are stored as object metadata, not sent as headers #513
Description
The S3 File Writer's Custom HTTP Headers table says "These headers will be included on all PUT requests (writing objects).", but the engine passes every custom header to the AWS SDK as user metadata (PutObjectRequest...metadata(getCustomHeaders()) in S3Connection.doWriteFile, and the same in move when copying between buckets). S3 stores each one as x-amz-meta-<name>, so none of them changes the object.
Measured on main and on #506 against real S3, with one File Writer carrying these custom headers:
| Header | What S3 did |
|---|---|
Content-Type: text/plain |
content type stays application/octet-stream |
Content-Encoding: identity, Content-Disposition: attachment; filename="report.txt", Cache-Control: max-age=60, Content-Language: en, Expires: Thu, 01 Jan 2099 00:00:00 GMT |
not set |
x-amz-storage-class: STANDARD_IA |
storage class stays STANDARD |
x-amz-server-side-encryption: aws:kms |
encryption stays the bucket default, AES256 |
x-amz-tagging: source=oie |
no tags |
Every one of them showed up as user metadata instead (content-type: text/plain, cache-control: max-age=60, x-amz-storage-class: STANDARD_IA and so on). A plain GET of the object returns Content-Type: application/octet-stream, no Cache-Control or Content-Disposition, and our values only as x-amz-meta-* headers. Headers meant as metadata work: x-amz-meta-foo: bar is stored as foo: bar.
History, from the code (not tested): before the AWS SDK v2 upgrade in 6f0afcf (first released in 3.9.0), S3Connection added these headers with request.putCustomRequestHeader(...). That commit changed it to .metadata(...). NextGen's original spec for S3 mode, nextgenhealthcare/connect#4148, lists "Custom HTTP headers".
A fix would send these as their request fields and keep the rest as metadata. That would change what existing channels write: the content-type metadata value they produce today would go away.
Found while fixing #511.
Activity