Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Scratch" storage #2061

Open
cat-bro opened this issue Jul 12, 2024 · 0 comments
Open

"Scratch" storage #2061

cat-bro opened this issue Jul 12, 2024 · 0 comments

Comments

@cat-bro
Copy link
Collaborator

cat-bro commented Jul 12, 2024

Not to be confused with /mnt/scratch, the fast disk that job working directories are on. This is a new kind of user data directory where files would be routinely cleared out after a fixed time period (e.g. 2 months). Users would be allowed a greater volume of this kind of storage because of its temporary nature.

(1) Try this out on dev
Clear some space by removing older working directories and files (data is not safe on dev)
Add a scratch storage directory to the dev object store and a method of clearing it (check what EU and Main have done for this)
Practice using it, see how users can elect a default storage type, move things between temporary/normal storage, choose storage for a history. Screenshots of this can be used in the documentation of this feature that will be provided to users.

(2) Add this to production
For the sake of file transfer through an upload store folder (nginx or tus), all writable user data directories should be on the same volume

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant