In April, Talkstandards.com hosted an open forum on the topic of formal review/ranking of Standard Setting Organisations (SSO). Contributors were asked to discuss whether such review/ranking of SSOs would foster or impede efficient standardization?
Ajit Jaokar argued that any “standards for standards” effort risked the creation of a “class system between consortia” based on an arbitrary set of criteria and will only be more difficult to apply as new innovation becomes increasingly cross-domain (e-health, Mobile health, etc).
Helen Disney, arguing that the priority of an SSO should be promotion of both competition and innovation, identified the criticism that formal review creates more bureaucracy within the standard setting organisations and as such may slow change in dynamic markets such as the IT sector. However, she also presented the countervailing argument that rankings may provide valuable information that reduces search costs and facilitates the connecting of business with SSOs, and so long as SSOs continue to foster competition and innovation, rankings may be a valuable exercise.
Daminen Gerdin highlighted the significant challenges inherent in the ranking of SSOs, particularly the difficulty in determining first the requisite criteria and secondly the weight to place on each of these criteria in computing any final measure of rank. He further questioned whether such rankings would not do more harm than good. Geradin suggested that rankings may negatively affect diversity and competition between SSOs, by causing poorly ranked SSOs to disappear for the standards setting landscape.
Mattias Ganslandt discussed the 2008 IBM guidelines for SSO membership, which set out a set of criteria required for IBM to take part in an SSO. Ganslandt argued that policies such as this work towards a standardization of the standardization process and may have negative effects on innovation. Forcing SSOs, which differ from each other in many ways, to comply with a set of criteria would not benefit stakeholders, especially not the end-users.
Knut Blind argued that the crucial aspect with respect to ranking SSO is the selection and weighting of the criteria and that such a complex issue would require a broad range of criteria to be integrated. He stressed however that there exists inherent difficulty in defining unique and quantifiable parameters, e.g. for openness.
Steve Mutkoski warned of the dangers of placing “form over substance”, and forgetting the distinction between a successful standard and a successful/“highly rated” SSO. He argued that the main challenge associated with ranking SSOs was not the defining of ranking criteria, but that a single set of criteria and formula through which to pass these criteria will never be able to accurately assessment and compare all SSOs. Furthermore it will miss the intended mark due to its failure to guarantee that individual standards themselves be of high quality, relevant and obtain market acceptance.
Daniel Dardailler (who related his argument to his experience with W3C) suggested that a formalized review process could potentially foster efficient standardization, so long as the process of developing the review criteria and methodology were open and publicly available. Such ranking he claimed could aide stakeholders with their investment choices and allow new consortia gain immediate support provided they follow the criteria. Conversely, Jay Kesan argued that formalized rankings are more likely to hinder standardization than foster it as the criteria developed may be based on factors not common to all SSOs.
