Federation: Not an Answer to Pan-Enterprise Search

Many organizations have attempted to provide pan-enterprise search using a federated search infrastructure, where queries are federated out to the native search engines of the many disparate repositories in the enterprise. While this may be an appealing quick fix, this actually opens the enterprise up to significant risk.

Firstly, by federating search out to native search engines, one assumes that they are capable of effectively searching their own data. Given that many repositories rely on out-dated, end-of-life search products for their native search, this is not a reasonable assumption to make. These search engines rarely search all information in the repository, instead using sleight of hand techniques to perpetuate the illusion of performance. This is a significant problem now that the FRCP mandate that all Electronically Stored Information (ESI) be made searchable. Federated approaches therefore do not guarantee compliance.

Secondly, federation does not necessarily produce a single, coherent, relevance ranking. Each search engine to which a query is federated determines relevance ranking independently and each engine will return a separate list of results. Compiling these results into one coherent list is not trivial. Not only will each engine have determined the order of the returned results using different methods, but the weightings assigned to different results will also vary between search engines. Most legacy search engines, even if they are able to federate results out to other search engines, have no ability to intelligently handle the results they get back.

Finally, federation is not a scalable solution. Relying on native search engines means that an accurate results list cannot be built until one has all the results from all the source repositories. Consequently, the speed at which the results list is returned is dependent on the speed of the slowest engine. Furthermore, since every query has to be sent to all of the native search engines in a federated environment, this greatly increases not only network traffic, but also load on the repositories themselves. When this problem is combined with common legacy security techniques (such as unmapped methods) the network traffic demanded by even a modest intranet search can be crippling.

Autonomy's Approach

In contrast, Autonomy is able to directly search over 1,000 different content formats across 400 different content repositories, guaranteeing that all information has been searched whilst respecting security. Federation is not required which provides significant benefits in terms of performance and scalability, and ensures that the enterprise is FRCP compliant. Should federation be desired, Autonomy can provide this capability; however for full pan-enterprise deployments, federation is not recommended.

Further Reference:

The Evolution of Search

Forthcoming Events

Archived Events

There do not seem to be any upcoming events related to this page at the moment, please visit our seminar and webinar pages for more information.

There do not seem to be any case studies related to this page available yet, however please visit our publications site at http://publications.autonomy.com/ for more information.