Scalability

We sat down with Alan Gilmore, CTO of Meridio to discuss how Meridio is addressing the scalability needs of organizations.

John: Alan, every vendor touts their product as scalable. What then is scalability and why is it an issue at all?

Alan: In general, scalability of an eDRM (enterprise Document and Records Management) system is the ability to increase its capacity more or less indefinitely by adding more hardware. Organizations need to feel confident that their eDRM vendor can meet any scalability challenge they face. The worst possible eventuality would be an eDRM system growing in quantity of information stored, number of users, or usage and then failing because the system itself cannot scale to meet one or another of these challenges.

John: How does Meridio scale then?

Alan: There are several levels of scaling to be considered.

One of the easiest ways of scaling is just putting the system on higher capacity hardware. That usually means leveraging multiple processors and large memory capacity. Good software design is needed to be able to handle many transactions in parallel, as they often use the same resources. Subtle bottlenecks are common, leading to a "serialization" effect, which prevents the software getting the expected increase in performance. We have spent countless hours running systems flat out to detect and remove these effects from Meridio.

John: Well, that seems easy enough. But there is only so much you can do with one server. It will eventually run out of power. What is the next logical step?

Alan: The next step is to add servers to a system.

This is called multi-server scaling, and is when the system is distributed over a number of separate machines to share the load. The key to this is the use of "stateless" components, which do not need to communicate with each other in order to do their job. This could be compared with adding additional counters in a bank as the number of customers coming through increases. If this statelessness can be achieved, there is almost no limit to the capacity of a system, as more and more servers can be added side by side.

As Meridio customers will know, stateless server architecture has been a design principle of Meridio from the very beginning.

John: Those options sound great for a single site. How do you scale if the organization has multiple sites, or worst case, locations all over the world?

Alan: That type of scaling is called geographical scaling. We use this term for user populations that are distributed widely across the globe. The principal new challenges this brings are to do with network limitations. As well as bandwidth limits, the "round trip" latency for requests from the client to the server becomes significant. This could be a twentieth of a second for a transatlantic round trip, for example. It becomes very important to keep the number of round trips very small for each user action.

Unfortunately, the rich user interfaces typically found in client-server systems often rely on a lot of communication to give them their responsiveness, in contrast to the typical browser application that sacrifices richness for long-distance efficiency. It is this paradox that has led us down the path of Smart Clients, which aim to provide the best of both worlds in this area.

John: What happens if an organization needs fast response times around the globe? What is the best solution for that environment?

Alan: When you need fast response times around the globe for multiple users and sites, you need to use Replication.

Replication allows information to be held in multiple distributed systems, and automatically kept in step. This gives users in each location the ability to access the system locally. This has been a major area of investment for us, partly driven by military deployments, where such requirements are particularly frequent and tough. As well as fast local response, it can greatly reduce the overall bandwidth used by the system.

John: How is scalability different for Collaboration vs. eDRM?

Alan: Collaboration is typically decentralized, with users sharing information in small collaboration areas set aside for a specific project or task. The number of such areas can be very large - look at a large SharePoint deployment, for example - but there is usually a small number of documents or users in each area. There can be big scaling challenges, but they are very different to the scalability challenges of eDRM systems where the number of documents involved is often very large.

Collaboration systems may struggle when a very large number of documents need to be held in a single workspace or library. That is why, for example, some customers choose to deploy a combination of SharePoint and Meridio, giving scalability across the lifecycle, from early authoring through to long term preservation.

You can find out more about Meridio scalability here.