Nextcloud experience
-
Hello everybody!
A month ago, we decided to move our shared files to Nextcloud (mostly from Dropbox)
We partially used NC for almost 2 years with few users (first installed on premise, later hosted).
Now we moved all 40 users, with 181.000 files (ca. 96GB of data).
Most files are in group folders, not personal.
We mostly use NC offline (with Linux and Windows clients), not online .
On 4 of our sites, we have fileservers that are registered as NC users, and their NC folders are shared in local LANs.Last week we somehow lost 3 group folders with >2.000 files and we had trouble with restoring files. (https://mangolassi.it/topic/22897/nextcloud-how-to-restore-many-deleted-files/43)
We have following important issues that we have not noticed in our test period with few users:
- Some subfolders are deleted, and we can't find them in "Deleted files" in NC (theoreticaly possible that somebody used "Delete permanently" after deletion, but hghly unlikely that that happened)
- Online Search function in NC, does not work for some users (and work for others on same PC/browser)
- There is now way to easy restore large number of (accidentaly) deleted files in NC (like we used to in Dropbox) - discussed in linked thread above)
We need to reconsider our choice of Nextcloud.
What is your experience with Nextcloud?
Do you have experience with usage of this size (180.000 files, 40 users, many group folders) and do they have similar problems?
Do you have similar problems or better experience with Dropbox, Onedrive, Google Backup and Sync... (with usage on this scale)? -
We also had few serious issues with NC's client for Windows.
- Client app did not worked on Intel UHD graphics for almost 2 years (solved after some Windows update, but Google and Dropbox clients never had that issue)
- Client app stop syncing after each change of wifi/LAN network (neede to be manually restarted)
These issues are solved late last year, but toghether with these new issues, I am afraid to rely on NC now.
-
Are you using NextCloud as a backup? If not, where is your backup?
All of our Nextcloud data is backed up to S3 and our S3 buckets are Cross-Region Replicated using "Cross-Region Replication".
Restoring is not a simple task, but we don't lose sleep over the prospect of losing data; because it is backed up... twice.
-
@Mario-Jakovina said in Nextcloud experience:
We mostly use NC offline (with Linux and Windows clients), not online .
huh - you sync 96 GB of data to 40 users for mainly offline use? And you don't have conflict issues (more than one person editing a file while offline?
Plus - MAN, that's a lot of data to replicate everywhere.
OK OK OK - before JB jumps on me - that's not exactly what you said, but the details are pretty light.On 4 of our sites, we have fileservers that are registered as NC users, and their NC folders are shared in local LANs.
Is the expectation of these 4 sites to use the files in an online state? Is the setup this way because of need/desire for faster access?
-
@Dashrender said in Nextcloud experience:
@Mario-Jakovina said in Nextcloud experience:
We mostly use NC offline (with Linux and Windows clients), not online .
huh - you sync 96 GB of data to 40 users for mainly offline use? And you don't have conflict issues (more than one person editing a file while offline?
Plus - MAN, that's a lot of data to replicate everywhere.
OK OK OK - before JB jumps on me - that's not exactly what you said, but the details are pretty light.On 4 of our sites, we have fileservers that are registered as NC users, and their NC folders are shared in local LANs.
Is the expectation of these 4 sites to use the files in an online state? Is the setup this way because of need/desire for faster access?
The amount of data and the number of users doesn't really matter. It's the bandwidth that matters. How much data is changed every day, how many people needs access to that data, how fast do they need that access and how big is the pipe?
40 people are not all sharing the same files with each other and are not all working on the same thing.
For instance the average file size in Mario's case is 0.5 MB. Assume 40 user each change 40 files every day and every file is shared by 10 people on average. Using a cloud service that means you have 40x40x0.5=800MB to upload every day and 40x40x10x0.5=8GB to download everyday. 8GB per day to download is 1GB/hour or 300 kB/sec during the day. So you need about 3 Mbits/sec download speed to dedicate for file transfer and 0.3 MBit/sec for uploads.
Most companies could easily support that amount of data traffic from one site. And in this case it's several.
And even with this low bandwidth requirement it would only take a few seconds to upload your average 0.5MB file. And a few seconds before it's downloaded to another computer.
-
In this case I think a cloud service with no local fileservers and data replication directly to the users local drives makes a lot more sense than Nextcloud. It simplifies everything.
We use Zoho Workdrive and the smallest package is 2 EUR / user / month. That would be 80 EUR / month for 40 users and you'd have 4TB of storage space to share.
Files are local from the users perspective so nothing for them to do besides their usual work.
-
@Pete-S said in Nextcloud experience:
@Dashrender said in Nextcloud experience:
@Mario-Jakovina said in Nextcloud experience:
We mostly use NC offline (with Linux and Windows clients), not online .
huh - you sync 96 GB of data to 40 users for mainly offline use? And you don't have conflict issues (more than one person editing a file while offline?
Plus - MAN, that's a lot of data to replicate everywhere.
OK OK OK - before JB jumps on me - that's not exactly what you said, but the details are pretty light.On 4 of our sites, we have fileservers that are registered as NC users, and their NC folders are shared in local LANs.
Is the expectation of these 4 sites to use the files in an online state? Is the setup this way because of need/desire for faster access?
The amount of data and the number of users doesn't really matter. It's the bandwidth that matters. How much data is changed every day, how many people needs access to that data, how fast do they need that access and how big is the pipe?
40 people are not all sharing the same files with each other and are not all working on the same thing.
For instance the average file size in Mario's case is 0.5 MB. Assume 40 user each change 40 files every day and every file is shared by 10 people on average. Using a cloud service that means you have 40x40x0.5=800MB to upload every day and 40x40x10x0.5=8GB to download everyday. 8GB per day to download is 1GB/hour or 300 kB/sec during the day. So you need about 3 Mbits/sec download speed to dedicate for file transfer and 0.3 MBit/sec for uploads.
Most companies could easily support that amount of data traffic from one site. And in this case it's several.
And even with this low bandwidth requirement it would only take a few seconds to upload your average 0.5MB file. And a few seconds before it's downloaded to another computer.
Yeah I get all that - though my question is - how many of those files really need to sync back to the PCs?
I'm also interested in learning about your setup - you seem to have the NC sync client on all 40 endpoints, but then you also have sync clients on these four servers. Do they sync different data? Is the server sync'ed data only available to the users when they are in the office accessing the local share? Also, why is that data synced to NC? is that the backup? or at least one part of it? These four servers - is the data the same on all four? your description didn't seem to make that case - it appears that each server has it's own account, and it's own respective data for that location.
-
@Mario-Jakovina said in Nextcloud experience:
Do you have experience with usage of this size (180.000 files, 40 users, many group folders) and do they have similar problems?
We have about that many users, but far fewer files, here internally. We've had people accidentally delete things similarly, but on a smaller scale. They were able to recovery from the "Trash Bin" method, in that example case.
-
@Dashrender said in Nextcloud experience:
huh - you sync 96 GB of data to 40 users for mainly offline use? And you don't have conflict issues (more than one person editing a file while offline?
The bigger it is, the less likely you are to get those issues.
-
@Dashrender said in Nextcloud experience:
And you don't have conflict issues (more than one person editing a file while offline?
Most filetypes don't have conflict issues. That's pretty unique to working offline with Word, Excel, etc. Not that other things can't have conflicts, they do. But the workflow common to most document types protects against that just from how they are used. And if you don't have people collaborating on documents, which is actually surprisingly rare for most shops, it's not a big deal.
-
@Pete-S said in Nextcloud experience:
We use Zoho Workdrive and the smallest package is 2 EUR / user / month. That would be 80 EUR / month for 40 users and you'd have 4TB of storage space to share.
We are using the $3/u/m WorkPlace that comes with WorkDrive and it's been really good thus far.
-
@scottalanmiller said in Nextcloud experience:
@Dashrender said in Nextcloud experience:
And you don't have conflict issues (more than one person editing a file while offline?
Most filetypes don't have conflict issues. That's pretty unique to working offline with Word, Excel, etc. Not that other things can't have conflicts, they do. But the workflow common to most document types protects against that just from how they are used. And if you don't have people collaborating on documents, which is actually surprisingly rare for most shops, it's not a big deal.
I agree. though collaboration in the past was challenging - which lead to multiple copies of the same files, with multiple versions, etc.
We have no idea what the files in question are (at least I don't recall him saying). -
@Pete-S said in Nextcloud experience:
@Dashrender said in Nextcloud experience:
@Mario-Jakovina said in Nextcloud experience:
We mostly use NC offline (with Linux and Windows clients), not online .
huh - you sync 96 GB of data to 40 users for mainly offline use? And you don't have conflict issues (more than one person editing a file while offline?
Plus - MAN, that's a lot of data to replicate everywhere.
OK OK OK - before JB jumps on me - that's not exactly what you said, but the details are pretty light.On 4 of our sites, we have fileservers that are registered as NC users, and their NC folders are shared in local LANs.
Is the expectation of these 4 sites to use the files in an online state? Is the setup this way because of need/desire for faster access?
The amount of data and the number of users doesn't really matter. It's the bandwidth that matters. How much data is changed every day, how many people needs access to that data, how fast do they need that access and how big is the pipe?
40 people are not all sharing the same files with each other and are not all working on the same thing.
For instance the average file size in Mario's case is 0.5 MB. Assume 40 user each change 40 files every day and every file is shared by 10 people on average. Using a cloud service that means you have 40x40x0.5=800MB to upload every day and 40x40x10x0.5=8GB to download everyday. 8GB per day to download is 1GB/hour or 300 kB/sec during the day. So you need about 3 Mbits/sec download speed to dedicate for file transfer and 0.3 MBit/sec for uploads.
Most companies could easily support that amount of data traffic from one site. And in this case it's several.
And even with this low bandwidth requirement it would only take a few seconds to upload your average 0.5MB file. And a few seconds before it's downloaded to another computer.
Pretty sure this isn't what happens with cloud services. For instance from what I've seen, dropbox will dedupe files and store metadata locally.
Then if it's already deduped they can just copy the diff.
-
@stacksofplates said in Nextcloud experience:
@Pete-S said in Nextcloud experience:
@Dashrender said in Nextcloud experience:
@Mario-Jakovina said in Nextcloud experience:
We mostly use NC offline (with Linux and Windows clients), not online .
huh - you sync 96 GB of data to 40 users for mainly offline use? And you don't have conflict issues (more than one person editing a file while offline?
Plus - MAN, that's a lot of data to replicate everywhere.
OK OK OK - before JB jumps on me - that's not exactly what you said, but the details are pretty light.On 4 of our sites, we have fileservers that are registered as NC users, and their NC folders are shared in local LANs.
Is the expectation of these 4 sites to use the files in an online state? Is the setup this way because of need/desire for faster access?
The amount of data and the number of users doesn't really matter. It's the bandwidth that matters. How much data is changed every day, how many people needs access to that data, how fast do they need that access and how big is the pipe?
40 people are not all sharing the same files with each other and are not all working on the same thing.
For instance the average file size in Mario's case is 0.5 MB. Assume 40 user each change 40 files every day and every file is shared by 10 people on average. Using a cloud service that means you have 40x40x0.5=800MB to upload every day and 40x40x10x0.5=8GB to download everyday. 8GB per day to download is 1GB/hour or 300 kB/sec during the day. So you need about 3 Mbits/sec download speed to dedicate for file transfer and 0.3 MBit/sec for uploads.
Most companies could easily support that amount of data traffic from one site. And in this case it's several.
And even with this low bandwidth requirement it would only take a few seconds to upload your average 0.5MB file. And a few seconds before it's downloaded to another computer.
Pretty sure this isn't what happens with cloud services. For instance from what I've seen, dropbox will dedupe files and store metadata locally.
Then if it's already deduped they can just copy the diff.
I was calculating worst case without any regard to how the files are synchronized.
You could have peer to peer sync with the LAN, you could have block based transfer and only send the blocks that have changed as you mentioned and you could also have files on demand, which means that they are only synced when needed. In any case it will lead to less information being transferred.
The point of my post was to show that even without any fancy schmancy features the amount of data and bandwidth requirements are pretty low.
-
I will try to answer most of your questions in one post:
@JasGot said in Nextcloud experience:
Are you using NextCloud as a backup? If not, where is your backup?
Nextcloud is not backup. We have backup of it stored in Dropbox. However earlier when we used Dropbox,it was easier to undo accidental deletes (mostly accidental moves) then to restore files from backups. We have recovered now from backups, but I'm worried about what happened in Nextcloud.
@Dashrender said in Nextcloud experience:
huh - you sync 96 GB of data to 40 users for mainly offline use? And you don't have conflict issues (more than one person editing a file while offline?
Plus - MAN, that's a lot of data to replicate everywhere.Users used to notice conflicts and solved them. Unfortunately they have not used to use files online (mostly becuse they used to use offline MSOffice, and in some cases it was a requirement becuase of VBA macros)
We do not sync all files with all accounts, of course not.@Dashrender said in Nextcloud experience:
Is the expectation of these 4 sites to use the files in an online state? Is the setup this way because of need/desire for faster access?
On these 4 sites, Linux fileservers serve Nextcloud group folders as local SMB shares, and they also sync them to Nextcloud so that users outside of LAN can see them or edit them if they need. It is setup this way because we had unreliable internet until few years ago, and because of VBA macros that we need to use offline...
@Dashrender said in Nextcloud experience:
Yeah I get all that - though my question is - how many of those files really need to sync back to the PCs?
I'm also interested in learning about your setup - you seem to have the NC sync client on all 40 endpoints, but then you also have sync clients on these four servers. Do they sync different data? Is the server sync'ed data only available to the users when they are in the office accessing the local share? Also, why is that data synced to NC? is that the backup? or at least one part of it? These four servers - is the data the same on all four? your description didn't seem to make that case - it appears that each server has it's own account, and it's own respective data for that location.Yes, we have 40 users/accounts and four of them are servers. Yes they all sync at least partly different data, but most data are used by more then one user/account/endpoint (some have read only access, some can write/delete...).
Data that servers sync, are accessible by some people from other sites or laptops.
Yes, all servers have own account and it's own data, but they hold their "local" data in their "group folder" in Nextcloud. But those group folders are also accessible to at least some other Nextcloud users (that's why they need to be synced). We also have some local folders that do not to be synced, and they are not in Nextcloud -
@scottalanmiller said in Nextcloud experience:
@Mario-Jakovina said in Nextcloud experience:
Do you have experience with usage of this size (180.000 files, 40 users, many group folders) and do they have similar problems?
We have about that many users, but far fewer files, here internally. We've had people accidentally delete things similarly, but on a smaller scale. They were able to recovery from the "Trash Bin" method, in that example case.
Number of our files is ridiculous. Together with Dropbox, that we still use, we have almost 300.000 files.
We probably need to reorganize our use, sort old files if they need to be archived, and identify and delete old junk files. But we also don't have organizational experience with how to organize this "cleaning" and how to establish system to avoid clutter in future. -
@Pete-S said in Nextcloud experience:
In this case I think a cloud service with no local fileservers and data replication directly to the users local drives makes a lot more sense than Nextcloud.
What is this "cloud service" that does "data replication directly to the users local drives" if Nextcloud is not that? We use local fileservers just to decrease number of file conflicts.
-
@Pete-S said in Nextcloud experience:
We use Zoho Workdrive and the smallest package is 2 EUR / user / month. That would be 80 EUR / month for 40 users and you'd have 4TB of storage space to share.
Files are local from the users perspective so nothing for them to do besides their usual work.ZohoDrive sounds fine (5x cheaper then Dropbox)
Is it functionaly different from Dropbox/Nextcloud?
Nextcloud/Dropbox is also "local from user perspective" so I don't understand what you mean by that? -
@Mario-Jakovina said in Nextcloud experience:
ZohoDrive sounds fine (5x cheaper then Dropbox)
Is it functionaly different from Dropbox/Nextcloud?It's different, for sure. They all vary some. But it's very similar.
https://www.zoho.com/workdrive/desktop-sync.html
My team uses it without the sync tool. We almost exclusively use either Zoho Docs (because that's our document format) or PDFs which don't require collaboration and are mostly archival. But if we were working with lots of document types, we'd use it with the sync tool.
-
@Mario-Jakovina said in Nextcloud experience:
What is this "cloud service" that does "data replication directly to the users local drives" if Nextcloud is not that? We use local fileservers just to decrease number of file conflicts.
When I say local I mean that the actual file is residing on the users harddrive. So if you unplug the network, the file can still be accessed.
Since you are talking about fileservers, that distinction has to be made. Because if the files are on the fileserver and you have an SMB share or something else, it's not on the users local drive.
