mirrors/garage
1
0
Fork 0
mirror of https://git.deuxfleurs.fr/Deuxfleurs/garage.git synced 2026-05-15 13:46:54 -04:00
Code Issues Releases Packages Wiki Activity
604 commits 30 branches 89 tags 51 MiB
Commit graph

604 commits

This branch
This branch
All branches
Author SHA1 Message Date
Quentin Dufour
88781b61ce Update compatibility target 2022-01-24 17:50:16 +01:00
Quentin Dufour
c435004d9f Reword connect 2022-01-24 13:59:25 +01:00
Quentin Dufour
3e07f10255 Add WinSCP and update menu 2022-01-24 13:52:53 +01:00
Quentin Dufour
11f54e2b8b Add support for duck 2022-01-24 13:44:39 +01:00
Quentin Dufour
a8ef761731 Add docker registry 2022-01-24 12:37:12 +01:00
Quentin Dufour
ba7be3f895 Add doc for Publii + Peertube 2022-01-24 12:04:58 +01:00
Quentin Dufour
9374389f87 Add tests for CORS 2022-01-14 11:47:27 +01:00
Alex Auvolat
bed3106c6a
 		Implement {Put,Get,Delete}BucketCors and CORS in web server 2022-01-13 17:27:16 +01:00
Alex Auvolat
60c0033c8b
 		Update documentation 2022-01-13 14:25:22 +01:00
Alex Auvolat
d4dd2e2640
 		Make use of website config, return error document on error 2022-01-13 14:25:19 +01:00
Alex Auvolat
9eb211948e
 		Allow setting index document and error document on the CLI 2022-01-13 14:25:19 +01:00
Alex Auvolat
3ea8ca1b9e
 		Implement GetBucketWebsite 2022-01-13 14:23:52 +01:00
Alex Auvolat
f7349f4005
 		Add quotes in returned etags 2022-01-13 14:03:33 +01:00
Alex Auvolat
1ee8f596ee
 		Testing for UploadPartCopies and bugfixes in AWS signatures 2022-01-13 14:03:30 +01:00
Alex Auvolat
6617a72220
 		Implement UploadPartCopy 2022-01-13 13:58:47 +01:00
Alex Auvolat
3770a34e3d
 		Implement x-amz-copy-if-xxx copy preconditions and return more headers on copy (fix #187) 2022-01-13 13:56:55 +01:00
Quentin
b4592a00fe Implement ListMultipartUploads (#171) 
Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2.

It took me some times as I wanted to propose the following things:
  - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476).
  - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator)
  - An IO closure to make unit tests possibles.
  - Unit tests, to track regressions and document how to interact with the code
  - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK.

Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications.

Some points that we might want to discuss:
  - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation.
  - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token.
  - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now).

Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/171
Co-authored-by: Quentin <quentin@dufour.io>
Co-committed-by: Quentin <quentin@dufour.io>
 2022-01-12 19:04:55 +01:00
Alex Auvolat
9cb2e9e57c
 		Add documentation to migrate to 0.6 2022-01-10 14:42:58 +01:00
Alex Auvolat
3586c7257c
 		Clarify error message 2022-01-10 14:10:04 +01:00
Alex Auvolat
17ea28a438
 		Fix trivial bug in CLI 2022-01-10 12:38:33 +01:00
Alex Auvolat
8f39360f22
 		Update documentation 2022-01-07 17:14:37 +01:00
Alex Auvolat
7ee11f0eb6
 		Fix unit tests 2022-01-05 17:34:48 +01:00
Alex Auvolat
168a90dfb5
 		Fix some error codes 2022-01-05 17:07:36 +01:00
Alex Auvolat
fb1e31add0
 		Small CLI changes 2022-01-05 16:28:46 +01:00
Alex Auvolat
135858d067
 		Implement DeleteBucket 2022-01-05 16:28:19 +01:00
Alex Auvolat
8395030e48
 		Implement CreateBucket 2022-01-05 15:56:48 +01:00
Alex Auvolat
9431090b1e
 		Implement key allow|deny --create-bucket 2022-01-05 15:12:59 +01:00
Alex Auvolat
677ab60cc1
 		Small changes in key model and refactoring 2022-01-04 18:59:17 +01:00
Alex Auvolat
df35feba18
 		New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updates 2022-01-04 12:53:14 +01:00
Alex Auvolat
1bcd6fabbd
 		New buckets for 0.6.0: small changes 
- Fix bucket delete

- fix merge of bucket creation date

- Replace deletable with option in aliases
    Rationale: if two aliases point to conflicting bucket, resolving
    by making an arbitrary choice risks making data accessible when it
    shouldn't be. We'd rather resolve to deleting the alias until
    someone puts it back.
 2022-01-04 12:52:47 +01:00
Alex Auvolat
ba7f268b99
 		Rename and change query filters 2022-01-04 12:52:46 +01:00
Alex Auvolat
de37658b94
 		Hopefully fix Nix build 2022-01-04 12:52:46 +01:00
Alex Auvolat
e59c23a69d
 		Refactor logic for setting/unsetting aliases 2022-01-04 12:52:46 +01:00
Alex Auvolat
2140cd7205
 		Remove website redirects 2022-01-04 12:52:46 +01:00
Alex Auvolat
beeef4758e
 		Some movement of helper code and refactoring of error handling 2022-01-04 12:52:46 +01:00
Alex Auvolat
d8ab5bdc3e
 		New buckets for 0.6.0: fix model and migration 2022-01-04 12:47:28 +01:00
Alex Auvolat
c7d5c73244
 		Add must_use to some CRDT functions 2022-01-04 12:47:28 +01:00
Alex Auvolat
b76d0580a0
 		Fix forgotten flag 2022-01-04 12:47:28 +01:00
Alex Auvolat
87121dce9d
 		New buckets for 0.6.0: documentation and build files 2022-01-04 12:47:06 +01:00
Alex Auvolat
b1cfd16913
 		New buckets for 0.6.0: small fixes, including: 
- ensure bucket names are correct aws s3 names
- when making aliases, ensure timestamps of links in both ways are the
  same
- fix small remarks by trinity
- don't have a separate website_access field
 2022-01-04 12:46:41 +01:00
Alex Auvolat
5db600e231
 		More complete output to bucket info and key info 2022-01-04 12:46:41 +01:00
Alex Auvolat
4d30e62db4
 		New buckets for 0.6.0: migration code and build files 2022-01-04 12:46:13 +01:00
Alex Auvolat
0bbb6673e7
 		Model changes 2022-01-04 12:45:52 +01:00
Alex Auvolat
53f71b3a57
 		Implement bucket alias and bucket unalias 2022-01-04 12:45:51 +01:00
Alex Auvolat
5b1117e582
 		New model for buckets 2022-01-04 12:45:46 +01:00
Alex Auvolat
8f6026de5e
 		Make table name a const in trait 2021-12-15 15:39:10 +01:00
trinity-1686a
945b75dbf1 update s3 compatibility list (#177) 
Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/177
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 15:05:54 +01:00
trinity-1686a
ca7b438f3f less strict ListBuckets (#178) 
fix #175

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/178
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 15:05:36 +01:00
trinity-1686a
1eb972b1ac Add compression using zstd (#173) 
fix #27

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/173
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 11:26:43 +01:00
trinity-1686a
60d4459926 BucketWebsite (#174) 
fix #77

this does not store anything but a on/off switch for website, and does not implement GetBucketWebsite as it would require storing more. GetBucketWebsite should be pretty easy to implement once data is stored though.

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/174
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 10:41:39 +01:00