Method of data management for efficiently storing and retrieving data to respond to user access requests

DETAILED DESCRIPTION The disadvantages associated with the prior art are overcome by the present invention of a server comprising a plurality of modules, each of which contains a single processor and a cluster of, for example, 16 disk drives, and a host controller that communicates with and assigns data requests to each of the modules.
Data is written to the disk drives by striping the data across the 16-disk drive cluster of a single module according to a RAID-5 protocol, with parity and spares distributed amongst the disk drives in the cluster.
The architecture of the present invention employs dynamic data management methods, which determine whether data should reside on disk or secondary storage, on which disk drives data should be stored, and how data should be replicated and/or migrated to new disk drives based on observed user access patterns.
These methods also migrate popular data to faster disk tracks to reduce average access time and thus improve performance.
User access requests are assigned to modules based on the data stored at each module, and each module's current load (the number of requests waiting to be serviced).
If the requested data is not on a disk drive, the data is retrieved from secondary storage, and may be stored on the disk drives for rapid subsequent access.
When a requested data item on the disk drive is replicated, load balancing is performed by assigning the request to the module holding the data with the lowest load.
In addition, user access requests waiting to retrieve replicated data may be dynamically and seamlessly migrated to another module based on changes in module loads.