mirrors/garage
1
0
Fork 0
mirror of https://git.deuxfleurs.fr/Deuxfleurs/garage.git synced 2026-05-15 13:46:54 -04:00
Code Issues Releases Packages Wiki Activity
858 commits 30 branches 89 tags 51 MiB
Commit graph

858 commits

This branch
This branch
All branches
Author SHA1 Message Date
Alex Auvolat
6dab836f3a
 		Multipart improvements 
- support part_number for HeadObject
- add checks in complete_multipart_upload
 2022-01-24 21:04:40 +01:00
Alex Auvolat
513a6b15f9
 		Handle OPTIONS on website endpoint 2022-01-24 12:32:28 +01:00
Alex Auvolat
ea7fb901eb
 		Implement {Put,Get,Delete}BucketCors and CORS in general 
- OPTIONS request against API endpoint
- Returning corresponding CORS headers on API calls
- Returning corresponding CORS headers on website GET's
 2022-01-24 11:58:00 +01:00
Trinity Pointard
820924534a use clamp instead of min(max()) 2022-01-24 11:56:59 +01:00
Quentin Dufour
94f0e7c135 Test ListParts endpoint with awscli 2022-01-21 10:42:35 +01:00
Quentin Dufour
440374524b Implement ListParts 2022-01-21 10:42:30 +01:00
Quentin Dufour
fe003d6fbc Add ListPartsResult structure 2022-01-20 16:38:55 +01:00
trinity-1686a
e55fa38c99 Add date verification to presigned urls (#196) 
fix #96
fix #162 by returning Forbidden instead Bad Request

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/196
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2022-01-18 12:22:31 +01:00
trinity-1686a
178e35f868 refactor s3_router and api_server to make unused Endpoint parameters more obvious 2022-01-17 15:50:24 +01:00
Alex Auvolat
7c049f1c94 Fix extreme value to be less extreme so that integration test works on 32bits 2022-01-17 12:56:29 +01:00
Jill
fdcddbe168 Fix Multipart Upload with WinSCP (#164) (#193) 
Closes #164.

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/193
Co-authored-by: Jill <kokakiwi@deuxfleurs.fr>
Co-committed-by: Jill <kokakiwi@deuxfleurs.fr>
 2022-01-17 11:18:40 +01:00
Jill
b45dcc1925 Support STREAMING-AWS4-HMAC-SHA256-PAYLOAD (#64) (#156) 
Closes #64.

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/156
Co-authored-by: Jill <kokakiwi@deuxfleurs.fr>
Co-committed-by: Jill <kokakiwi@deuxfleurs.fr>
 2022-01-17 10:55:31 +01:00
Alex Auvolat
60c0033c8b
 		Update documentation 2022-01-13 14:25:22 +01:00
Alex Auvolat
d4dd2e2640
 		Make use of website config, return error document on error 2022-01-13 14:25:19 +01:00
Alex Auvolat
9eb211948e
 		Allow setting index document and error document on the CLI 2022-01-13 14:25:19 +01:00
Alex Auvolat
3ea8ca1b9e
 		Implement GetBucketWebsite 2022-01-13 14:23:52 +01:00
Alex Auvolat
f7349f4005
 		Add quotes in returned etags 2022-01-13 14:03:33 +01:00
Alex Auvolat
1ee8f596ee
 		Testing for UploadPartCopies and bugfixes in AWS signatures 2022-01-13 14:03:30 +01:00
Alex Auvolat
6617a72220
 		Implement UploadPartCopy 2022-01-13 13:58:47 +01:00
Alex Auvolat
3770a34e3d
 		Implement x-amz-copy-if-xxx copy preconditions and return more headers on copy (fix #187) 2022-01-13 13:56:55 +01:00
Quentin
b4592a00fe Implement ListMultipartUploads (#171) 
Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2.

It took me some times as I wanted to propose the following things:
  - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476).
  - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator)
  - An IO closure to make unit tests possibles.
  - Unit tests, to track regressions and document how to interact with the code
  - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK.

Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications.

Some points that we might want to discuss:
  - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation.
  - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token.
  - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now).

Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/171
Co-authored-by: Quentin <quentin@dufour.io>
Co-committed-by: Quentin <quentin@dufour.io>
 2022-01-12 19:04:55 +01:00
Alex Auvolat
9cb2e9e57c
 		Add documentation to migrate to 0.6 2022-01-10 14:42:58 +01:00
Alex Auvolat
3586c7257c
 		Clarify error message 2022-01-10 14:10:04 +01:00
Alex Auvolat
17ea28a438
 		Fix trivial bug in CLI 2022-01-10 12:38:33 +01:00
Alex Auvolat
8f39360f22
 		Update documentation 2022-01-07 17:14:37 +01:00
Alex Auvolat
7ee11f0eb6
 		Fix unit tests 2022-01-05 17:34:48 +01:00
Alex Auvolat
168a90dfb5
 		Fix some error codes 2022-01-05 17:07:36 +01:00
Alex Auvolat
fb1e31add0
 		Small CLI changes 2022-01-05 16:28:46 +01:00
Alex Auvolat
135858d067
 		Implement DeleteBucket 2022-01-05 16:28:19 +01:00
Alex Auvolat
8395030e48
 		Implement CreateBucket 2022-01-05 15:56:48 +01:00
Alex Auvolat
9431090b1e
 		Implement key allow|deny --create-bucket 2022-01-05 15:12:59 +01:00
Alex Auvolat
677ab60cc1
 		Small changes in key model and refactoring 2022-01-04 18:59:17 +01:00
Alex Auvolat
df35feba18
 		New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updates 2022-01-04 12:53:14 +01:00
Alex Auvolat
1bcd6fabbd
 		New buckets for 0.6.0: small changes 
- Fix bucket delete

- fix merge of bucket creation date

- Replace deletable with option in aliases
    Rationale: if two aliases point to conflicting bucket, resolving
    by making an arbitrary choice risks making data accessible when it
    shouldn't be. We'd rather resolve to deleting the alias until
    someone puts it back.
 2022-01-04 12:52:47 +01:00
Alex Auvolat
ba7f268b99
 		Rename and change query filters 2022-01-04 12:52:46 +01:00
Alex Auvolat
de37658b94
 		Hopefully fix Nix build 2022-01-04 12:52:46 +01:00
Alex Auvolat
e59c23a69d
 		Refactor logic for setting/unsetting aliases 2022-01-04 12:52:46 +01:00
Alex Auvolat
2140cd7205
 		Remove website redirects 2022-01-04 12:52:46 +01:00
Alex Auvolat
beeef4758e
 		Some movement of helper code and refactoring of error handling 2022-01-04 12:52:46 +01:00
Alex Auvolat
d8ab5bdc3e
 		New buckets for 0.6.0: fix model and migration 2022-01-04 12:47:28 +01:00
Alex Auvolat
c7d5c73244
 		Add must_use to some CRDT functions 2022-01-04 12:47:28 +01:00
Alex Auvolat
b76d0580a0
 		Fix forgotten flag 2022-01-04 12:47:28 +01:00
Alex Auvolat
87121dce9d
 		New buckets for 0.6.0: documentation and build files 2022-01-04 12:47:06 +01:00
Alex Auvolat
b1cfd16913
 		New buckets for 0.6.0: small fixes, including: 
- ensure bucket names are correct aws s3 names
- when making aliases, ensure timestamps of links in both ways are the
  same
- fix small remarks by trinity
- don't have a separate website_access field
 2022-01-04 12:46:41 +01:00
Alex Auvolat
5db600e231
 		More complete output to bucket info and key info 2022-01-04 12:46:41 +01:00
Alex Auvolat
4d30e62db4
 		New buckets for 0.6.0: migration code and build files 2022-01-04 12:46:13 +01:00
Alex Auvolat
0bbb6673e7
 		Model changes 2022-01-04 12:45:52 +01:00
Alex Auvolat
53f71b3a57
 		Implement bucket alias and bucket unalias 2022-01-04 12:45:51 +01:00
Alex Auvolat
5b1117e582
 		New model for buckets 2022-01-04 12:45:46 +01:00
Alex Auvolat
8f6026de5e
 		Make table name a const in trait 2021-12-15 15:39:10 +01:00