The notion of creating standards for standards setting organizations not a new one, but it certainly has attracted more discussion and interest over the past 18 months. The challenge I see with this movement is less in coming up with the types of broad criteria that would be helpful to individual participants in the standards setting ecosystem, there are a host of useful research projects that have done just that over the years. Instead, my concern is related to the idea that there is a single exhaustive set of criteria and moreover a single formula through which those criteria can be passed to create an assessment or comparison of SSOs. Moreover, whatever a systematic or formulaic process such as this might inform us about the SSO itself, I am concerned that it doesn’t guarantee anything with respect to several of the key attributes of individual standards themselves, namely that they be of high quality, relevant and most importantly that they obtain market acceptance.
When we talk about “standards participants” we are really using a broad brush- we mean firms who invent the technology that goes into standards (“innovators”), firms that build standards into products (“implementers”) and end users of the product and standards within it (“users”). Researchers often layer other groups of interested parties over these participants– including the government or the general public– although these other interested parties are not always participants in the sense that they show up at technical committees and participate in the creation of a standard. While the utopian goal of SSOs is to create policies that result in “balance” among these interests, ultimately balance means different things to different people in different situations. One author has very clearly pointed out how differing business models among firms create differing incentives for various SSO participants. As the old phrase goes “you say tomato, I say tomahto.”
Given the range of participants and interested parties, the criteria or attributes that one could use to assess SSOs are necessarily quite broad. In 1998, the US government was one of the first to set forth a list of attributes which were used to define “voluntary consensus standards bodies” in OMB Circular A-119. Those attributes included “openness,” “Balance of interest,” “Due process,” “An appeals process” and “consensus.” Several years later, the WTO’s Committee on Technical Barriers to Trade adopted a set of principles which speak to a similarly wide range of matters relating to fairness in standard setting, including transparency and consensus. More recently, the European Commission enumerated a similar set of attributes in a whitepaper entitled “Modernising ICT Standardisation in the EU – The Way Forward.”
Where the real challenge lies though, is in taking those generally accepted attributes forward and creating a certification scheme or an assessment system based upon how an SSO approaches each of these attributes of its process. A very thoughtful and informative study by IDC illustrates the challenges of trying to establish a ranking system for even a single one of these attributes. That study examined whether there was a way to “evaluate the degree of openness of the leading standards setting organizations” and it incorporated under the concept of “openness” many of the attributes mentioned so far, including transparency, due process, access to documents and so on. In the course of applying these attributes to ten SSOs, the study made several key conclusions, including these:
• “It is difficult to see any clear patterns in the ratings”
• “[T]he concepts of openness and consensus have been implemented using different models that relate to the type of organization, their formal foundation and their degrees of formalization”
• “It can be… difficult to make a distinction of which form of ’openness’ is the most appropriate”
• “[the] concept [of “openness”] has been implemented in different ways in different standards setting organizations which renders comparisons difficult”
Just by way of illustration, the study pointed out that for one attribute– openness of membership– it might be tempting to just focus on cost of membership, since lower membership fees would result in a perception of more openness. Such limited funding, however, “means fewer activities in cost intensive areas like interoperability testing,” which are of course important for the long term success of the standard. These findings from the IDC study are interesting and lend strong support to the contention that the various SSO attributes we have been discussing are not amenable to a comparison system which is then aggregated across the range of attributes to attempt to provide a meaningful distinction among SSOs.
Despite the demonstrated impracticability of such a system, it is possible that “Part 2” of BSI PAS 98 could wind up being used to do just that. Part 1 “represents a codification of good practice and procedures for the establishment and operation of consortia” which one would assume will largely reflect the previously mentioned SSO attributes. Part 2 is entitled “Evaluating standards related consortia”. While BSI may disclaim any intent for the evaluation process in Part 2 to be used to construct some type of certification program, examples abound of past standards where unplanned certification schemes materialized.
At the end of the day, it seems to me that we shouldn’t put form over substance: We should not lose sight of the distinction between a successful standard and a successful (or “highly rated”) SSO. History has taught us that even the seemingly “best” SSO is capable of producing standards which ultimately see little or no uptake in the marketplace. Likewise, high quality and market winning standards have emerged out of what we might charitably call less than open and inclusive processes (some of which might not initially have even involved something that we would even call an SSO). One the best examples of this complex dynamic between form and substance is the emergence of TCP/IP from an informal collaboration among researchers to win out over a competing standard ISO/IEC 7498-1:1994 (more commonly known as the OSI Model) that was subject to far greater formalities. If we rigidly apply this list of SSO attributes we clearly run the risk of ending up in a situation where we might “force” the market to use a standard from a “better” SSO, in effect that would mean the “next OSI” might prevail over the “next TCP/IP.”
