Saving and copying can fail for files on NFS due to outdated attribute cache

[krassowski]

Description

Loosing data is never good. Currently jupyter-server does not have any special provision to handle permission or other file system errors due outdated attributes for NFS, SSHFS, GlusterFS, CephFS etc.

Reproduce

Try to write to a notebook that was opened when read-only but since the permissions were updated on the remote FS. See that copy2_safe routine fails with PermissionError:

    def copy2_safe(src, dst, log=None):
        """copy src to dst
    
        like shutil.copy2, but log errors in copystat instead of raising
        """
        # if src file is not writable, avoid creating a back-up
        if not os.access(src, os.W_OK):
            if log:
                log.debug("Source file, %s, is not writable", src, exc_info=True)
>           raise PermissionError(errno.EACCES, f"File is not writable: {src}")
E           PermissionError: [Errno 13] File is not writable: /tmp/nfs_mount/6zxrq928/file.txt

Expected behavior

Saving and copying files is possible without waiting potentially a few minutes before the attributes cache refreshes.

Context

• Operating System and version: Ubuntu
• Jupyter Server version: 2.17.0