Basic ideas about Storage in Media Industry

In any media industry, the basic storage has to be designed in such a way to create model file, store it in centralized location, manage and edit at different levels of animation online so as to deliver accurate digital media output.
For media applications, the latest technology updates is with the help of cluster computing, animation software, and high performance network storage in Giga network architecture. This architecture provides higher performance and better accessibility of data at any given point of time with low cost and with higher performance.
The central storage environment helps this industry to keep alive the data at all the time and make them visible to all the end users to access it concurrently- especially online without disruption.
So what should be the nature of the Storage Box in the Media Industry?
Below are few key points to be taken into consideration before we plan & implement and Storage.

Finding a hi-performance solution
When the bunch of HDDs generally a “JBOD” connected to a system and configured with RAID is known as “Storage Box”
Creating a storage sub system for the animation industry is not an easy task.
The Storage Admin responsible for storage needs to be aware and know the answers to the following basic queries.
• How flexible your storage is going to be?
• How many concurrent sessions it can handle?
• How much redundancy does it have?
• How to assure that the data stored is safe?
• How sure are you that the planned storage meets your expectation?
• How to get your ROI by investing in your solution?
Every Engineer should know the above simple FAQs.

There are multiple queries to be addressed to make a good and better storage solution with higher performance.

Few tips to analyze before designing storage for any Organization:

1. Define capacity- depends on the requirement.
2. Even if it’s little old technology the data should be safe with redundancy.
3. Plan for backup.
4. Calculate your concurrent sessions hitting the storage, calculate your throughput values.
5. Select the proper storage which meets all your requirements without compromise.
6. Select the proper vendor who can get your solution in selective manner.
7. Go for a solution with total support system to avoid discrepancies.
8. Plan for disaster recovery.
9. Plan the budget in advance before taking decisions.
10. Plan the architecture – coordinate with multiple vendors- select the one which can match the maximum requirements.
11. Need to have “SLA” between your organization and the vendor. This agreement will give importance to :

o Shipping and delivery of the storage box at the premises without any damage.
o Hardware connections with the proper naming on storage devices as well as the cables (Ethernet & FC) used
o Configuration & Implementation.
o Training to be imparted regarding the setup. (Certification if required)
o Having certain important Hardware’s (like HDDs, Memory chips, FC cables, HBA adapters etc) onsite to swap incase of failures.
o 24x7 Support either through telephone or onsite if required.
o Patches & updates in terms of HBA, Firmware, Bios etc at regular intervals without disturbing the setup.
o Backup policies maintenance.
o Disaster Plans
o Single point of contact options instead of multiple calls for support

What’s next?

Once you have the storage in place –
1. Deploying the storage
2. Serving mode of data (CIFS, NFS, Native protocol)
3. Bandwidth & Throughput calculations
4. Segregate the file systems with block sizes – stripe breadth – page size according to the projects
5. Pipeline or the workflow fine-tuning
6. Project time frame
7. Space allocation for projects
8. Block size analyze for project file sizes
9. IOPs calculations
10. Benchmarks with Pull Push testing
11. Network optimization for better data flow
12. Data Backup & Archive windows specifications
13. Rendering
14. Seamless operation of online expansion
15. Disaster Recovery Plans
16. Regular maintenance of patches & firmware upgrades
17. Optimizing the internal cache of controllers & server nodes

Advantages of central storage

1. Data can be accessed from various departments across locations
2. Higher performance
3. I/O Multiple Long or Short Thread Read
4. Caching On the server for Read Performance
5. Data Stripe Across Disk and Across Sub-System
6. Easy data flow once you make the perfect pipeline channel
7. The higher throughput central storage having the advantage to have a single point of managing & backup the media files not only for the administrators’ but also for the project pipeline
8. Hi-end fabrics make sure that the data flows in multi-paths in order to avoid bottlenecks
9. Flexible Storage will make it easy for the end user in accessing file systems online
10. Cluster storage obviously gives failover and fail safe
11. The storage can handle the High performance compute clusters read & write files from the central location with peak load even at peak time access from end users without creating bottlenecks
12. Multipath access from the FC channels helps the users to access the data without interruption or network chock –even at the time of backend FC port failures
Analyze the perfection on the storage decision. Do not compromise technology for anything.