Using AWS managed File Systems with Workbench
In multi-node configurations of Workbench, shared file storage is a requirement for users’ Linux home directories and a convenience for users’ shared data. Commonly, this shared storage is provided by a mounted NFS volume.
AWS provides two main options to use managed file storage:
EFS is a partially NFS 4.0 and 4.1 compatible managed file system. The Posit Team documentation contains detailed information on how to use Amazon EFS with Posit Team)
FSx is a class of cloud services that provides access to multiple AWS managed shared file storage solutions, such as Lustre or OpenZFS.
This article provides information on the compatibility of Posit Team with Amazon EFS, FSx for Lustre and FSx for OpenZFS managed file storage services with specific focus on
- shared home directories
- shared user data
- Project Sharing
For completeness sake, we also compare to a traditiona NFSv3 or NFSv4 mount as well.
Summary of Findings
Overall, sharing user data and home directories works well with NFS, Amazon EFS, Amazon FSx for Lustre, and Amazon FSx OpenZFS.
EFS and FSx for OpenZFS lack default ACL support, therefore Project Sharing is not supported for these file systems.
The below table summarizes our results
Provide shared user data | Provide shared home directories | Compatible with Workbench Project Sharing | |
---|---|---|---|
Mounted NFS | ✅ | ✅ | ✅ |
Amazon EFS | ✅ | ✅ | ❌ |
Amazon FSx Lustre | ✅ | ✅ | ❌ |
Amazon FSx OpenZFS | ✅ | ✅ | ❌ |
A Note on Project Sharing
In this article we refer to Project Sharing as the Workbench feature that allows collaborative and simultaneous editing of project files from a shared storage location. In order for Project Sharing to function, the underlying file system must support POSIX access control lists (ACL), and notably, the ability to define a default ACL.
Collaborative editing can be a useful tool for pair programming, however it is important to note that collaborators can edit the same file concurrently and edits are not tracked. A more robust method for collaborative development is to utilize a version control system (e.g., git
). Version control provides a different workflow for jointly working on a project where changes to a file are tracked, approved, and merged. Such workflows are more asynchronous than the Project Sharing feature implemented in Workbench, but more robust for collaborative development. In cases where the Project Sharing feature as implemented in Workbench is not supported due to the use of the managed file storage, use of version control system will still provide means for collaboration.