Marketing - Video Editing Storage
-
@IRJ said in Marketing - Video Editing Storage:
What kind of marketing department is this? A movie firm? It seems insane to have video files average 10-20gb. Even high budget commercials are probably only that size.. Is this a bunch of templates or something?
I would be interested in understanding their work flow, as it seems very extravagant to have such large files and so many of them?
Raw video is easily that large. It's not final cut and transcode that it becomes manageable.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
I am being asked to find a storage solution for our video editing department. They use various Adobe tools on Mac clients.
Sounds like my every day life. I use MacOS and Apple editing products and produce an insane amount of material daily.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
Marketing said speed was the main concern. Accessing the large files is currently slow causing many delays. They have so far been sharing USB 3.0 devices between each other, without backups.Traditionally, this is one of the use cases for SAN. Large post processing shops (this is marketing, but more or less the same use case) typically use high speed SAN with clustered file systems to deal with this issue.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
Our CIO is now pushing for cloud only solutions for the storage where marketing can check in/out the files they need, killing my NAS idea. I have concerns on this but am open, so would like some advice. What have you used or what would you suggest to use to provide this?
I am concerned that when an editor wants to do something with the 300 GB file, that will have to pull down from the cloud, bringing our WAN link to a crawl until the download finishes. Then, once they have finished editing, they have to upload the end product, again causing bandwidth issues. This is exacerbated shoudl multiple editors be pulling multiple files at the same time. Even at 500 Mbps with no overhead that is around 1.5h of time to sit and wait for a 300 GB file whilst everybody else is affected on the same WAN link.Yes, that would be a huge performance bottleneck. I have a similar situation here. I do this a lot and it is only me, a single user. I end up waiting most of a day for files to upload and download. I literally have my other thread on here discussing how to move files to prep them for final publication because this issue is so extreme.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
I assume the'video editing' cloud storage providers are like OneDrive or DropBox, where each machine has a local cache so do not have to pull from cloud each time. But, wouldnt that mean that our video editing workstations all need to have a 20 TB local drive to store this local cache?
Kind of all over the place on this one, any ideas folks?Yes, in order to have the files ready all of the machines would need to keep a copy AND would need to download changes as they happen.
-
@Pete-S said in Marketing - Video Editing Storage:
It's unrealistic to assume you can use your entire bandwith for one download. It's also completely unrealistic to assume you can get 500 Mbps sustained from whatever cloud storage you have.
My cloud storage from a video provider gives me about 65Mb/s in real world bandwidth, if that helps.
-
@IRJ said in Marketing - Video Editing Storage:
What kind of marketing department is this? A movie firm? It seems insane to have video files average 10-20gb. Even high budget commercials are probably only that size.. Is this a bunch of templates or something?
Check your Olympus EM-1 MK2 for video, it'll produce that amount fast (but breaks up the files, but just imagine that it kept them as a single file like newer cameras do.)
If you are shooting with a Fuji XH2 or similar, you'll produce 20GB files every few minutes.
I'm just on a GoPro 11 and all my files are 10GB because that's its file limit. If the GoPro kept the files together, I'd routinely have 20GB - 40GB files. And that's low bit rate compared to serious cameras. Imagine if you shot on Panasonic GH6 or better, especially if you go to 6K or 8K! And I'm using H.265 which is tiny. If you shot on ProRes, which is what most do, the file sizes will explode quickly. 100GB or more would be nothing.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
If their PCs accessing a NAS at 1-10Gbps isn't good enough because their primary concern is speed, why would they push for way slower cloud storage, assuming no on-prem cache?
-
@Obsolesce said in Marketing - Video Editing Storage:
@Jimmy9008 said in Marketing - Video Editing Storage:
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
If their PCs accessing a NAS at 1-10Gbps isn't good enough because their primary concern is speed, why would they push for way slower cloud storage, assuming no on-prem cache?
I archive my video in the cloud, but I would not want to work from it without a local cache.
-
@Obsolesce said in Marketing - Video Editing Storage:
@Jimmy9008 said in Marketing - Video Editing Storage:
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
If their PCs accessing a NAS at 1-10Gbps isn't good enough because their primary concern is speed, why would they push for way slower cloud storage, assuming no on-prem cache?
1 - 10 Gbps would be more than fine. That is what I proposed. But, the CIO is asking that the storage is Cloud only. Leading to this issue where the editing workstations are in office and the storage is remote.
-
@scottalanmiller said in Marketing - Video Editing Storage:
@IRJ said in Marketing - Video Editing Storage:
What kind of marketing department is this? A movie firm? It seems insane to have video files average 10-20gb. Even high budget commercials are probably only that size.. Is this a bunch of templates or something?
Check your Olympus EM-1 MK2 for video, it'll produce that amount fast (but breaks up the files, but just imagine that it kept them as a single file like newer cameras do.)
If you are shooting with a Fuji XH2 or similar, you'll produce 20GB files every few minutes.
I'm just on a GoPro 11 and all my files are 10GB because that's its file limit. If the GoPro kept the files together, I'd routinely have 20GB - 40GB files. And that's low bit rate compared to serious cameras. Imagine if you shot on Panasonic GH6 or better, especially if you go to 6K or 8K! And I'm using H.265 which is tiny. If you shot on ProRes, which is what most do, the file sizes will explode quickly. 100GB or more would be nothing.
I've not seen the files first hand, am just told these are the sizes by that team. They are telling me the cameras will do 40 minutes to 1 hour recordings of presentations and stuff like that at 4k and 8k, and they are usually 300 GB - 400 GB.
-
@Dashrender said in Marketing - Video Editing Storage:
Sounds like all the bases have been covered.
It's unrealistic to work with files of that size (even 10 GB files are unrealistic to work on from cloud).
As Pete said - show the owners the time it takes to move the data, then show them the cost of a potential VDI solution with hosted storage - your solution of a NAS will quickly be back on the table with backups to cloud.
I have been doing some research and am interested in this company, they seem to have both storage in cloud and editing via browser. Any thoughts on this, or better alternatives?
Does anybody know if Adobe has a similar entirely cloud solution for video editing? When I go to their site I find it a mess and confusing.
-
This looks like another option, although, it does just look like a NAS to me, just through a specific 'media' vendor.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
@Dashrender said in Marketing - Video Editing Storage:
Sounds like all the bases have been covered.
It's unrealistic to work with files of that size (even 10 GB files are unrealistic to work on from cloud).
As Pete said - show the owners the time it takes to move the data, then show them the cost of a potential VDI solution with hosted storage - your solution of a NAS will quickly be back on the table with backups to cloud.
I have been doing some research and am interested in this company, they seem to have both storage in cloud and editing via browser. Any thoughts on this, or better alternatives?
Does anybody know if Adobe has a similar entirely cloud solution for video editing? When I go to their site I find it a mess and confusing.
Frankly, that site has no real useful information after a very quick look. It's all marketing fluff and no pricing listed anywhere, which tells me the price is crazy high. It might make sense, but most likely not.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
This looks like another option, although, it does just look like a NAS to me, just through a specific 'media' vendor.
That's a scam. I use those guys about once a week as an example of "market vertical scams." I've had customers get seriously screwed over by them.
Never buy "industry" IT equipment, it's always a scam. IT is IT, anything industry specific is another way of saying "not good enough to pass IT muster, so we try to bypass IT by claiming it's specifically made for an industry."
They literally make the worst storage you could possibly imagine.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
@scottalanmiller said in Marketing - Video Editing Storage:
@IRJ said in Marketing - Video Editing Storage:
What kind of marketing department is this? A movie firm? It seems insane to have video files average 10-20gb. Even high budget commercials are probably only that size.. Is this a bunch of templates or something?
Check your Olympus EM-1 MK2 for video, it'll produce that amount fast (but breaks up the files, but just imagine that it kept them as a single file like newer cameras do.)
If you are shooting with a Fuji XH2 or similar, you'll produce 20GB files every few minutes.
I'm just on a GoPro 11 and all my files are 10GB because that's its file limit. If the GoPro kept the files together, I'd routinely have 20GB - 40GB files. And that's low bit rate compared to serious cameras. Imagine if you shot on Panasonic GH6 or better, especially if you go to 6K or 8K! And I'm using H.265 which is tiny. If you shot on ProRes, which is what most do, the file sizes will explode quickly. 100GB or more would be nothing.
I've not seen the files first hand, am just told these are the sizes by that team. They are telling me the cameras will do 40 minutes to 1 hour recordings of presentations and stuff like that at 4k and 8k, and they are usually 300 GB - 400 GB.
Yup, that's what I would expect from something like a top end Fuji, Sony, Panasonic or Canon doing an hour of video.
-
@Jimmy9008 said in Marketing - Video Editing Storage:
@Obsolesce said in Marketing - Video Editing Storage:
@Jimmy9008 said in Marketing - Video Editing Storage:
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
If their PCs accessing a NAS at 1-10Gbps isn't good enough because their primary concern is speed, why would they push for way slower cloud storage, assuming no on-prem cache?
1 - 10 Gbps would be more than fine. That is what I proposed. But, the CIO is asking that the storage is Cloud only. Leading to this issue where the editing workstations are in office and the storage is remote.
I would connect the CIO with the users and ask if waiting hours or days to work on a file is good enough. Let the CIO take that up with users.
-
@Obsolesce said in Marketing - Video Editing Storage:
@Obsolesce said in Marketing - Video Editing Storage:
@Jimmy9008 said in Marketing - Video Editing Storage:
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
If their PCs accessing a NAS at 1-10Gbps isn't good enough because their primary concern is speed, why would they push for way slower cloud storage, assuming no on-prem cache?
I archive my video in the cloud, but I would not want to work from it without a local cache.
Right, I can't imagine how that could work. No matter how fast the pipe is, the latency would be too high. I don't even want to work via a SAN in the same office. I don't even want to work on a normal SSD. I use 4x NVMe for editing when possible, you really feel the difference and I'm only on 5.3K files in H.265, not 8K HDR on ProRes!!
-
@scottalanmiller said in Marketing - Video Editing Storage:
@Jimmy9008 said in Marketing - Video Editing Storage:
@Obsolesce said in Marketing - Video Editing Storage:
@Jimmy9008 said in Marketing - Video Editing Storage:
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
If their PCs accessing a NAS at 1-10Gbps isn't good enough because their primary concern is speed, why would they push for way slower cloud storage, assuming no on-prem cache?
1 - 10 Gbps would be more than fine. That is what I proposed. But, the CIO is asking that the storage is Cloud only. Leading to this issue where the editing workstations are in office and the storage is remote.
I would connect the CIO with the users and ask if waiting hours or days to work on a file is good enough. Let the CIO take that up with users.
The CIO is pretty much of the opinion that the user should plan better and download files they need for editing whilst working on something else.
-
@scottalanmiller said in Marketing - Video Editing Storage:
@Jimmy9008 said in Marketing - Video Editing Storage:
This looks like another option, although, it does just look like a NAS to me, just through a specific 'media' vendor.
That's a scam. I use those guys about once a week as an example of "market vertical scams." I've had customers get seriously screwed over by them.
Never buy "industry" IT equipment, it's always a scam. IT is IT, anything industry specific is another way of saying "not good enough to pass IT muster, so we try to bypass IT by claiming it's specifically made for an industry."
They literally make the worst storage you could possibly imagine.
Could you tell me more about this? I am not sure I fully understand but would like to. Is the thought that because its a "NAS for Editors" and not just a "NAS" that its not good, otherwise it would be a NAS for everybody regardless of need?
