Stanco's testimony was discussed on slashdot [5/1/03].
Who are these people? The New York City Council is what you would think, and the Select Committee on Technology & Government has been around for about a year, judging from its list of reports. Select committees, I would surmise, are called into existence for temporary periods, as opposed to standing committees which exist forever.
The Cyber Security Policy and Research Institute is a decade old (although until recently it was called the Cyberspace Policy Institute) and according to its mission statement "CPI's mission is to encourage, promote, facilitate, and execute interdisciplinary research in areas related to the nexus of society and the Internet." The Center of Open Source & Government was founded more recently by Tony Stanco; it sponsored a conference on open source and government last October (and another one in March).
Stanco's testimony lists seven reasons why open source in government is a good thing:
- Democratic Implications
- Privacy
- Cost
- Research and Development/Technology Transfer
- Education
- Job Creation
- Security
I discussed these in a post on slashdot. I'm not crazy about his arguments, but under Democratic Implications he did say: "Governments have special obligations to protect the integrity, confidentiality and accessibility of public information throughout time like no other entity in society. Therefore, storing and retrieving government data through secret and proprietary data formats tied to a single provider is especially problematic, since the usability, maintenance and permanence of government data should not depend on the goodwill or financial viability of commercial suppliers."
The Council was not debating a specific proposal on open source procurement in city government; it was a fact-finding mission. The briefing paper (PDF) is an excellent summary of the issues. As it states in its conclusion, one problem with open source laws is defining what open source means [OSS == open source software]:
The debate surrounding OSS, as well as government interventions into that decades long debate, coalesces around several competing factors that are not easily untangled – the commercialization of software, the presumed benefits of a competitive technology marketplace, security concerns, the interoperability of different technology systems, the role which government can and should play in promoting private sector business models as well as certain development models, to name just a few of the areas of interest. As the, albeit, carefully worded definition of OSS suggests, there is within these competing, sometimes complimentary arenas confusion even over what OSS is precisely....Statutory interventions, then, as evidenced in California as well as Oregon, represent undoubtedly complicated efforts inevitably predicated on a clear definition of OSS – something which even advocates for OSS have not been able to entirely stabilize themselves.
The briefing paper also has some good links at the bottom, including one to a white paper from February 1999 by Mitch Stoltz that may be the earliest one to call for government procurement of open source software (the Committee's briefing paper got its historical information from this paper). Stoltz mentions cost only briefly; his arguments for open source include security, intellectual property, Y2K issues, and its anti-monopolistic nature (this was during the thick of the Microsoft-DOJ lawsuit). He argues that government can support open source software either by purchasing it, or by encouraging its employees to become involved in developing it (an interesting point that I have not seen elsewhere). Unfortunately his only discussion of open standards relates to network protocols, not data formats.
One thing the Committee's briefing paper does not link to is a recent resolution put out by the Faculty Senate of the State University of New York at Buffalo (although Buffalo is nominally in the same state as New York City, I doubt many New Yorkers would care to acknowledge that fact). Titled Resolution for University Support of Open Software and Open Standards, it starts out well: "direct unmediated unfettered access to information is fundamental and essential to scholarly inquiry, academic dialog, research, the advancement of research methods, academic freedom, and freedom of speech" but then it wanders off into the standard anti-Microsoft weeds, complaining about Microsoft messing with standards such as Kerberos and Java, supporting Digital Rights Management, pushing aggressive licensing agreements on users, forcing upgrades, allegedly including spyware with Media Player, etc, etc.
The resolution does recover in the end, pointing out that Microsoft software can't be modified for research purposes, and includes the all-too-true statement: "the use of closed proprietary document formats and information management systems to store the work of faculty, students, and staff limits the ways these works can be accessed and archived, and jeopardizes access itself in the long term." It ends with a recomendation that SUNY Buffalo support GNU/Linux and OpenOffice specifically, and open source alternatives in general, and that "the Faculty of the University at Buffalo call on the University to implement a policy of promoting open document formats and communication protocols wherever possible and, in the case of broadcast announcements and other documents intended for a general audience, discouraging the use of secret and proprietary formats (such as Microsoft Word format) in favor of open formats (such as plain text or HTML) that are universally accessible."
The resolution was the brainchild of John Ringland, an associate professor in the math department who is also the chair of the Computer Services Committee of the Faculty Senate. Ringland is also opposed to the Digital Millennium Copyright Act and the Consumer Broadband and Digital Television Promotion Act (aka the "Hollings Bill") and favors the Digital Media Consumers' Rights Act, as you can see from this other resolution encouraging the university to lobby in favor of the DMCRA; a year earlier he posted a "call to action" and you can see earlier versions of both his open source resolution and his anti-DMCA/CBDTPA resolution. Ringland presented both his resolutions to the Senate [4/18/03]; the anti-DMCA/CBDTPA one was adopted on March 4, the open source one on April 1. And it is legitimate, despite the date.
Slashdot discussion of this resolution, for those interested, is here [4/2/03].
Since the committee was interested in gathering information, it did so, and as far as I know software procurement by the City of New York continues unchanged. You can read the full testimony if you wish; look on page 14 for the Initiative for Software Choice, a Microsoft-backed anti-open-source-law group. The testimony in favor of open source basically said "It works"; the testimony against basically said "A law requiring only open source software would be bad." Since these are not mutually exclusive, it's not clear what the Committee's takeaway would have been.
The debate was also meta-debated (briefly) on slashdot [12/11/02].
The first question you might ask yourself is: who are these people, what are the organizations they work for, and why should I care what they think about open source procurement laws?
The answer is that they all work for think tanks, research organizations that have more to do with government policy than you might think. A lot of the "experts" who appear on talk shows (political talk shows, I mean) are from places like this. They provide the intellectual backing for a lot of laws and they have the ear of government officials, so what they think and say is important. Anyone interested in passing laws involving open data formats (or open source for that matter) would be well served to get some think tank folks engaged in the debate.
Think tanks exist all over the political spectrum. There is a think tank, Capital Research Center, that studies other think tanks: you can see its quick summaries on the Cato Institute, the Competitive Enterprise Institute, and the Pacific Research Institute (the Progress and Freedom Foundation seems to have dropped off their radar, at least their free radar).
A quick scan of their mission statements reveals that the Cato Institute and the Pacific Research Institute are libertarian, and the Competitive Enterprise Institute and Progress and Freedom Foundation are conservative. Thus, according to the world's smallest political quiz, all four groups favor smaller government; the libertarians also believe that whatever government is left should leave its citizens alone. Conservatives, in general, tend to favor businesses, while libertarians tend to favor the individual.
Not surprisingly, DeLong, a conservative, comes out against open source laws [12/10/02]. His argument essentially echoes Microsoft's, although possibly not intentionally (typically, he says nothing about data formats). As he points out, he is not opposed to open source per se, merely laws that require it, and his summation is "if it ain't broke, don't fix it."
The two libertarians disagree: Sanchez is for open source laws [12/10/02], Arrison is against them [10/3/02] (Sanchez commented on both the DeLong piece [12/10/02] and the Arrison piece [10/9/02] in his blog, where he describes DeLong as also being a libertarian, although I have my doubts about that).
Arrison gives the standard libertarian argument: Open source software should succeed or fail in the market on its own merits, and the government should get out of the way. She makes an interesting comment: "But forcing the taxpayer's IT budget to favor one type of system over another for purely political reasons is wrong and antithetical to the spirit of the open source community." In other words, the open source movement is officially about giving users a choice (although of course for some people it is about beating Microsoft), so how can you favor a law that limits the government's choice?
Sanchez is more conflicted; as a libertarian he should be opposed to such laws, but he really likes open source software. He presents some basic technical arguments (faster bug fixes, proprietary software sitfling innovation), but then gets on his two main points. The first is that open source software procurement would be less prone to market-clogging lobbying (I'm not sure if this is true, given Red Hat's support of California's open source bill; it may simply be that the open source lobby is more politically naive and less politically active, the way Microsoft was about five years ago). Sanchez's second point, however, is about open data formats:
Proprietary software makers know that client data is locked up in a format they own. This places them in a unique position to provide upgrades, fixes, and other forms of technical support - especially when dealing with inertia bound bureaucracies less subject to the pressures that might make a private firm switch platforms more readily. They also know that software, like a VCR or fax machine, is often a "network good" characterized by "bandwagon effects," which make its value a function, not only of its intrinsic characteristics, but also of the number of other people using the same product. These facts taken together mean that firms can parlay government use of a proprietary format - PowerPoint, say - into sales of the client software to read it that format. Even when the client software is given away without charge, as with Adobe Acrobat Reader, firms know that if more users need to download their proprietary client in order to communicate with the government, that larger user base expands the market for their authoring software. Since companies can't expect to similarly capture those network benefits when producing open source software [emphasis added] - and, perhaps more importantly, needn't fear being locked out by a competitor who does - the stakes are far lower for any one contract. With proprietary software, government's potentially standard-setting procurement choices give it the role of market kingmaker.
If you just replace "open source" with "open data format" at the place I emphasized, you have a great argument for open data format laws.
Thus, the main difference between the two libertarians, Arrison and Sanchez, is that Sanchez recognized the benefits that the "open data format" part of an open source law would bring; Arrison does not. You can take Arrison's comments about why open source laws are bad, and Sanchez's comments about why open data formats are good, and combine them into one great argument that should convince any libertarian that open data format laws are an excellent idea.
Since this site is pushing for open data format laws, not surprisingly I have issues with both organizations.
I'll talk about the Initiative for Software Choice first, since it is the easiest to dismiss. Although it is nominally under the auspices of CompTIA, the Computing Technology Industry Association, there seems little doubt that it was nudged into existence by Microsoft following the open source bills introduced in 2002 (CompTIA is a legitimate industry group that runs the A+ certification program, among others).
The ISC comes across as a quickie lobbying group with not much intellectual heft behind it. For example its policy page complains about the Oklahoma open source bill being labeled an "emergency" bill, which is pure alarmism. It also lists an article by Tim O'Reilly [8/15/02]as being in support of the ISC, when he immediately posted a disclaimer at the top denying this, and later wrote a longer article explaining that he was on the side of Sincere Choice [9/27/02].
The ISC has four principles:
The first one is the main one: don't pass laws requiring open source only. The other three are related to Microsoft's concern that open source prevents a company from licensing its intellectual property: this was the main pillar of Microsoft's anti-Linux platform in the summer of 2002, and one of the arguments it used to lobby against open source laws.
I don't actually disagree with any of this (I am not concerned with the issue of licensing intellectual property, except as it relates to file formats), but the whole things seems so ham-handed and blatant that it makes Microsoft look like it is trying to hide something.
In response to the ISC, open source advocate Bruce Perens founded Sincere Choice. He explains his goals in his article announcing the founding of Sincere Choice [8/9/02]. He also attacks the ISC's goals, more shrilly than I think necessary.
The goals of Sincere Choice are:
The first point I support wholeheartedly: "In order to have a fair market, without customer "lock-in", file formats like those used by word processors must be open standards....We support reverse-engineering for purposes of compatibility, and oppose legislation that would restrict it." (An excellent point about reverse engineering, although hopefully if ODFI succeeds it will limit the need for that.)
Perens' second point mixes open file formats (which I support) with open network standards (which I am neutral on, and I think occur much more naturally than open file formats). His third point is just a slap at Microsoft, his fourth and fifth are aimed at the ISC's last three points about how to license university research (a side issue in my opinion), and his sixth is the only one that relates to open source laws: what he means is that governments should be free to set their own policy on software acquisition including passing a law requiring open source software. I think there is a difference between a government policy and a law, and I'm opposed to open source laws in general.
Both these organizations appear to be shells thrown up to support either side of the open source law debate (Perens, to his credit, admits that Sincere Choice is just that). The ISC does not mention open data formats, once again showing that this is a benefit of open source laws that nobody seems to have an argument against.
In any case, both sites and organizations appear to be dormant, although ISC did post some links in response to Oregon's open source law. In this sense Perens succeeded in damping the effect of ISC and gave an example of how to start up a one-man industry initiative, which I am of course interested in emulating.
However, it's not clear that "open" and "XML" are necessarily joined at the hip.
The debate is muddled because "open" data formats can mean different things to different people:
Data stored in XML is described using a schema, which defines what various tags mean. You could think of standard HTML as defining a schema, which every browser supports; with XML, anyone can define a schema.
An XML format would satisfy rule #1 and arguably satisfy #2, in the sense that XML schemas tend to be self-documenting. Microsoft's current binary format for Word satisfies none of those, so it is certainly not open. With ODFI I am trying to get Microsoft (and other companies) to satisfy #2 only--and even with XML formats I want companies to provide actual written documentation, not simply say "here is our schema, that's all the documentation you need".
Having a data format that satisfies #1 could be useful, but is not a requirement. And I am opposed to pushing for a data format, XML or other, that satisfies #3.
Microsoft has announced that Office 2003 is going to support storing data in XML. So that should make Scott McNealy happy, right? Well, not exactly. The article "At Microsoft's Mercy" [4/23/03] by Kendall Grant Clark captures some of the feelings about Microsoft's use of XML, from conspiracy theories that it is all a publicity scam, to those who think XML is over-hyped in any case.
Microsoft has defined one schema for Word, called WordML, but is also allowing users to define their own schemas in certain versions of Office. Will this help data interchange? As the Register puts it [4/25/03], "In the future, you may be faced with two flavors of nonsense. XML Word documents that have been mangled by Microsoft's XML-creation tools, and XML Word documents that have been mangled by users who add their own non-standard entities."
To really allow complete exchange of data between Word and other word processors, Microsoft would need to support not WordML, but a standard XML schema, one that satisfies rule #3 above. Many people seem to think that storing data in XML would automatically satisfy #3, based on the misperception that XML defines one overall standard schema for all data, or that computers would be able to automatically interpret the semantics of any XML schema. Others feel that doing XML "correctly" requires using a standard schema. Neither of these are true, as Microsoft has pointed out, and it apparently has no intention of supporting a standard schema.
The article "Why Standards?" [5/18/03] by Jim Waldo points out that standards that codify existing practice are much better than those that attempt to define something from the ground up. The problem with standards bodies is that they are slow and they can get political. If Microsoft wants to include a new feature in Word and therefore in its WordML schema, what should it do if the standards body that is certifying it a) takes too long to approve it or b) refuses to allow it altogether? Keep in mind that one of the main goals of ODFI is to allow information to be retrieved from a data file long after the program that reads it is gone. The key to this is having the format documented, and it doesn't matter if the documentation comes from one company or from a standards body.
I'll also point out that Microsoft is not going to make XML the default way to store data in Office 2003; the old .doc format will still be used unless the user choose to save as XML. Microsoft has to do this; otherwise, when one user in an organization upgrades to Office 2003 and starts producing XML documents, everyone else will have to upgrade at the same time or be left unable to read them. In fact Microsoft got roasted for causing this type of disruption when it changed its binary format between Word 95 and Word 97. The only way XML can become the default is to allow several versions of Office to ship that can all read XML; then perhaps in Office 2008 XML can become the default way to save files.
That is not to say that Microsoft's support of WordML has no benefits. To begin with, XML is text-based, not binary, so it is less susceptible to corruption, and a minor typo can be fixed with any editor (the same is true of the existing standard RTF). Also, as this post on XML-DEV [4/18/03] by John Cowan points out, most users do not crack open data formats themselves, but they do want third-party utilities that can do so. While it may take a little while for third parties to support a new flavor of WordML that accompanies a new version of Office, it is easier and more reliable for a third party to change its code to support reading a new XML schema than it is for them to reverse-engineer a new binary data format.
The meat of the law is one sentence: "No state agency nor the Purchasing Division of the Department of Central Services shall enter into a contract for the acquisition of computer software developed exclusively for the agency or the state [emphasis mine], unless the vendor agrees to provide to the agency or the state the source code for the software."
There's also a follow-up definition: "For purposes of this section, "source code" means the programming instruction for a computer program in its original form, created by a programmer with a text editor or a visual programming tool and saved in a file. "
It makes some sense to require this: if the government is paying for software, it arguably should get the source code also, in case the company that wrote the software won't support it, goes out of business, etc. Still this is a very simplistic law. Since it only applies to software written specifically for the government, it hardly qualifies as an "open source" law, and it went nowhere in the Oklahoma legislature.
The bill, HB 1627 (text is at the end of this article; RTF is here) was introduced by Republican Representative Mike Reynolds, a computer consultant and president of a software company. I would guess that Reynolds did not examine legislation that had been proposed in other states; the bill seems an attempt to write a law about open source without much consideration of the issues involved. I would classify is as well-intentioned, but hopeless in its current form.
The law has no preamble or explanation of "why?", so it's not clear what the goal is. Since it only states that the vendors must provide the source code and says nothing about the license under which the code must be provided, it appears that if Microsoft was writing software for the state of Oklahoma (which I suppose its consulting arm might do), the company could satisfy this bill by including Oklahoma in their "Shared Source" program. Thus, the bill does not give the cost benefits of true open source software, or the technical benefits of being able to modify and redistribute the code. It's not even clear if the code has to be provided in usable form (electronic, with build instructions). What would the State of Oklahoma hypothetically do if it was handed a printout of the 50 million (or whatever) lines of source code in Windows XP?
Of course, the law also does not specify anything about data formats, so the state would be left to decipher the source code if it wanted to figure those out. And the law wouldn't help the average citizen of Oklahoma, since it wouldn't apply to commercial software, and even if it did, the source code and any deciphered data formats would only be in the hands of the government.
Finally, the law applies a blanket rule to all software, and doesn't have any provision for being unable to find a vendor who will comply with it. Given the small number of companies that are likely to bid on contracts for custom-written software for the government, this needs to be addressed.
Despite this, Microsoft didn't like it. Of course the company is leery of the words "source code" and "voted into law" appearing too near each other in any legislative record anywhere, but still their attack on this law (through the Initiative for Software Choice's Policy Tracker, the last item) seems ham-handed. In particular they complain about the bill being labelled an "emergency", when in fact a quick scan of the Oklahoma legislative docket reveals that about half of all proposed bills are labelled that way, and it appears to simply be a legal device to cause the bill to take effect immediately after it is passed, instead of 90 days after the session ends (which I think is the default in Oklahoma). Luckily for humanity, the ISC promises to keep us posted if anything happens with the bill.
Anyway, the bill was introduced on January 15, 2003, and on the current Oklahoma bill tracking page (search for "1627") it is listed as "dormant" as of February 20.
The full text of the bill follows:
STATE OF OKLAHOMA
1st Session of the 49th Legislature (2003)
HOUSE BILL HB1627 By: Reynolds
AS INTRODUCED
An Act relating to state government; prohibiting contracts for certain computer software unless the source code is provided; requiring certain documentation; directing the State Purchasing Director to provide advice and assistance; providing definitions; providing for codification; and declaring an emergency.
BE IT ENACTED BY THE PEOPLE OF THE STATE OF OKLAHOMA:
SECTION 1. NEW LAW A new section of law to be codified in the Oklahoma Statutes as Section 85.7d of Title 74, unless there is created a duplication in numbering, reads as follows:
A. No state agency nor the Purchasing Division of the Department of Central Services shall enter into a contract for the acquisition of computer software developed exclusively for the agency or the state, unless the vendor agrees to provide to the agency or the state the source code for the software.
B. The State Purchasing Director or the procurement officer of state agencies not subject to the Central Purchasing Act shall not process any state agency request for the acquisition of computer software developed exclusively for the agency unless the proposed vendor provides documentation that complies with subsection A of this section.
C. The State Purchasing Director shall provide advice and assistance as may be required in order for state agencies to comply with the provisions of this section.
D. For purposes of this section, “state agency” shall include all state agencies, whether subject to the Central Purchasing Act or not.
E. For purposes of this section, “source code” means the programming instruction for a computer program in its original form, created by a programmer with a text editor or a visual programming tool and saved in a file.
SECTION 2. It being immediately necessary for the preservation of the public peace, health and safety, an emergency is hereby declared to exist, by reason whereof this act shall take effect and be in full force from and after its passage and approval.
Sam Steingold wrote "No Proprietary Binary Data Formats" [2/7/00, last updated 4/27/03]. He is making a slightly different point, that binary formats are bad (because you can't use tools like grep and diff). I agree with that statement, although it's not my goal with ODFI. For example he thinks PDF is bad and any XML format is good. He does not mention RTF (which is text-based and standardized).
Jeff Goldberg, with his "MS-Word is Not a document exchange format" [5/1/03], is against the use of Word attachments in particular, in a similar vein to Richard Stallman's complaint. He feels that a documented binary formats such as PDF is OK (which I agree with).
I'll throw in Scott McNealy's The "Case for Open File Formats" [10/12/01], even though he is Scott McNealy. As he writes, "Look at it this way: The data you put into a spreadsheet is yours. The content you put into a business presentation is yours. It's your intellectual property, right? So why would you allow any of it to be held captive in a proprietary file format?" Excellent point, although the article is really a plug for Sun's use of XML. I'm not sure XML is the magic solution. I'm sure someone could come up with extremely obfuscated XML if they wanted to. Plus, Microsoft can't make XML the standard "Save" format for a while, or it would require everyone in an organization to upgrade once a few people do (which is something Microsoft gets accused of doing with evil intent). What you really need is documentation (which Sun, to its credit, does have for its XML format).
Finally there is this article by Ramon Flores, which is in Portuguese, but has some good links at the bottom (including one to my original ODFI article). Actually there is a high degree of inter-linking between all the articles listed here.
What it interesting about these two bills is that they are being presented by their sponsors, and reported in the press, as "open source" bills. However, in some ways they are much closer to "open data format" bills, and as such may be the first of their kind.
Unfortunately, while the Texas bill was in committee it was replaced with a substitute bill that chopped it down to almost nothing, and now says nothing about open data formats. The Oregon bill was deemed unworkable in its current form and handed off to a working group for further discussion, although it is unclear how much discussion will actually take place.
CONTENTS
The bills
Suggested improvements
Current status
According to its history page on the Texas Legislature's site, the Texas Senate bill, sponsored by Republican Senator John Carona and titled "Relating to software acquisitions by state agencies," had its first reading on March 20; the Oregon House bill (the Oregon Legislature's bill status site does not provide unique URLs for search results), titled "Relating to software acquisitions by state government" and sponsored by Democratic Representative Phil Barnhart, had its first reading on March 5.
The Oregon bill was submitted at the suggestion of Ken Barber, a Microsoft Certified Systems Engineer from Eugene; the Texas bill is not identified as being the brainchild of any particular everyday citizen, and the general consensus is that it was based on the Oregon bill. In any case, the text of the bills is essentially identical, although the Oregon bill includes a preamble and the Texas bill goes on to amend some other sections of the government code to mention compliance with the new bill.
The bills are certainly promoted as open source bills. In Oregon, Representative Barnhart's site lists ten articles about the bill, all of which mention "open source" in the title. The EFF-Austin's site mentions open source repeatedly (EFF-Austin, incidentally, is independent of the main EFF).
Nonetheless, a closer reading of the bill reveals the fact that although they both require that the government only "consider" open source software, they state that the government should "avoid" products that do not use open standards.
The bills define five rules that the government must follow:
Open source is defined using the same six rules that were in the California bill, whch are the same six rules that were in the Peru bill:
Open standards, meanwhile, must be defined such that they:
This is a pretty reasonable definition; although it is written with an eye towards definitions produced by standards bodies, it does not require this; it states that that formats must be disclosed to the public, not just the government; and it is not particularly onerous, since any company could simply release their internal data format documentation for all to see and they would be in compliance (unless the part about "Do not favor one implementer over another" could be construed to mean "must not be defined by a single company that is also implementing the standard"). I might suggest tightening up the "Are available for all to read and implement" with some languages such as "and would allow one skilled in the art to fully interpret any data file using such an encoding."
Of the five rules set by the bills, item #1 (consider open source products) got the most press, and items #2 (acquire software on a value-for-money basis) and #3 (justify purchases of proprietary software) got the most flak from opponents. However, I think #4 (avoid the acquisition of products that do not use open standards) is the most interesting, and it's the only one (along with #5, avoid spyware) that sets a hard-and-fast rule on what the government should do. The fact that it was not the main thrust of opponents' arguments demonstrates that it is difficult to argue against the benefits of open data formats (as opposed to arguing against standards-based data formats, which is much easier).
It's not clear what the legal meaning of "avoid" is. Because the other thing that is on the "must avoid" list is spyware, which is a bit of a stretch to worry about, somehow in my mind this makes "avoid" seem more like "try to stay away from because it's just common sense" rather than "never buy", but I can see why legally "avoid" would be interpreted the latter way, and require the government to buy only software that supports open data formats.
As a result, while these bills are publicly touted as open source bills, they seem to be closer to open data format bills. The press seems to have missed this; writeups of the bills refer to only open source, and perhaps to open standards as a secondary goal. The Oregon bill was discussed, for example, by the Register [3/7/03] and Slashdot here [3/6/03], here [4/8/03] and here [4/18/03]; the Texas bill was discussed on Slashdot when it was submitted [3/15/03] and again when a hearing was held [5/10/03]. In all cases they were presented as bills that simply required consideration of open source.
The bills could certainly be improved by focussing them more on open data formats and downplaying or removing the open source requirements. The summary of the Oregon bill states that it "Requires state government to consider using open source software when acquiring new software. Sets other requirements for acquiring software." The "other requirements" (open data formats) should be expanded and emphasized.
The justification of open source in the preamble of the Oregon bill is fairly open to attack. For one thing, it hand-waves the cost issue by merely stating "The acquisition and widespread deployment of open source software can significantly reduce the state's costs of obtaining and maintaining software" which is of course extremely open to debate.
Meanwhile, it makes the point about open data formats: "It is necessary to the functioning of the state that computer data owned by the state be permanently available to the state throughout its useful life" but then rather than use that to point out the benefit of open data formats, it instead uses it as the lead-in to an attack on proprietary software manufacturers: "To guarantee the succession and permanence of public data, it is necessary that the state's accessibility to that data be independent of the goodwill of the state's computer system suppliers and the monopoly conditions imposed by these suppliers".
It then makes two statements: "Open source software guarantees that its encoding of data is not tied to a single provider" and "Open source software ensures interoperability through adherence to open, platform-neutral standards" which are both at worst false and at best an easy target for opponents to gum up the debate. The first one works much better if you simply substitute "Open data formats" for "Open source software", and although open data formats are not necessarily open and platform-neutral, they do go a long way towards ensuring interoperability.
The public hearings on the two bills had different outcomes.
According to Chris Sells, who testified against the bill [4/4/03] at the hearing (only because of provision #3, requiring justification of purchases of proprietary software, which would put too much of a burden on government employees), the result of the hearing was that the bill was referred to a working group "to come up with a bill that the committee could actually consider submitting for a vote". Chris also points out in his writeup that "open source didn't mean open standards or open data formats" (although the bill not just depend on "the source tells all" to guarantee documentation; it explicitly defines and requires open standards, in fact much more than it requires open source, as I have discussed).
The Oregon bill was scheduled for discussion at a House General Government Committee work session on April 17, but due to opposition it was passed over [4/18/03]. Along the way, provision #3 was dropped after opposition from industry and the government's central purchasing agency. According to the author of the bill, the Speaker of the House eventually killed the bill [5/2/03] due to "powerful out-of-state corporate interests."
In Texas, according to an account of the hearing [5/8/03] from Chip Rosenthal at EFF-Austin, "Sen. Carona has put forward a substitute bill. That means what we've been looking at as SB 1579 no longer exists. The substitute bill is much simpler. It just adds the following paragraph to Texas law:
The [Department of Information Resources] shall publish guidelines relating to the information a state agency must consider in determining whether the agency should acquire open source software products in addition to proprietary software products."
The text of the proposed bill is available here.
This law is about open source software, not free software, not surprising when Red Hat is helping promote it [8/14/02]. It appears to suffer from the fact that it is more market driven than philosophy driven, because it pays little attention to the benefit of open data formats. Tagged with the PR-friendly name "Digital Software Security Act", it does begin the preamble by stressing the need to "guarantee the succession and permanence of public software and data." However, it follows that with "it is necessary that the usability and maintenance of the software be independent of the goodwill of the suppliers, or on the monopoly conditions imposed by them." And just a bit later, "Proprietary software that can only be upgraded by the vendor creates an incentive for vendors to cease maintenance of older products for the purpose of forcing their customers to buy new products."
In other words, it comes across as essentially a bill of attainder against Microsoft. Data is at risk because your proprietary software vendor may charge too much for updates, provide you with software that is vulnerable to hackers, or grab your data and send it back to Redm--errr I mean wherever the proprietary software vendor happens to be headquartered.
The bill continues, "These goals necessitate that the encoding of data is not tied to a single provider. The use of standard and open formats in open source software gives a guarantee of this security and integrity access." Which is false, of course: just because data is stored in an open format does not mean it can't be hacked into or spied on. In any case, that's the last thing the law says about data formats; nowhere is it stated that documentation on formats have to be available, or that they have to be "standard" in any sense, beyond being a standard with one adherent, the program in question.
The law also briefly mentions cost, in the first sentence: "The State of California seeks to improve the security, interoperability and quality of its software while lowering the cost and invigorating competition among suppliers" but then drops that issue also so it can concentrate on slamming Microsoft.
The law was obviously written with the battle over Peru's law in mind. The preamble lists five things that the law does not do:
which are identical to five items listed in a letter written in defense of the Peru law.
The California law also takes the definition of "free software" in the Peruvian legislation and reproduces it verbatim as its definition of "open source software."
This proposal was a little too blatant to make much headway. One write proposed that it be called the "California Retaliatory Act Against Microsoft, Oracle and Other Companies That Are Way Too Big for Their Britches." Microsoft grumbled about it, and as far as I know it never obtained a sponsor in the Legislature and quietly went away. However, it could be viewed as the father of various open source bills that are making better headway in other states during the 2003 legislative session.
The first bill introduced was number 1609. The law was quite specific (and fairly restrictive) on what constituted free software. Although I do not like the requirement that only free software be used, the discussion in the bill did focus on open data formats as one of the primary benefits of free software.
Discussion of bill 1609 led to a FUD-ish letter [3/21/02] from Juan Alberto Gonzalez, the head of Microsoft Peru, to Edgar Villanueva Nunez, the Congressman who had introduced the Bill, which in turn inspired a long response [4/8/02] from Villanueva.
An article in Wired [4/22/02] covers the story, and you can also read an interview with Villanueva [5/24/02].
According to someone at GNU Peru, the law never went anywhere because of lobbying [7/27/02] by Microsoft and others.
Read on for discussion of the bill and the two letters.
CONTENTS
The original bill
Microsoft response
Response from Edgar Villanueva Nunez
Lobbying and Current Status
The original bill (translation)
The bill is straighforward in its objective: "Employ exclusively free software in all the systems and computing equipment of every State agency." Furthermore, it states:
The Executive, Legislative and Judicial branches, as well as the autonomous regional or local decentralized organisms and the corporations where the State holds the majority of the shares will use free software in their systems and computer equipment.
Although I do not approve of the proposed solution of requiring free software, the language in the explanation is fairly reasonable and is primarily concerned with issues that would be solved by requiring open data formats. As the bill says early on:
"The discovery of new information technologies, among them Free Software, has become an ideal instrument to assure the preservation of the State's data".
The text points out that the Peruvian constitution guarantees people access to data they need, while keeping personal information private. The principles the Bill wants to protect are summarized as:
Which are each expanded on in turn:
To guarantee the citizens' free access to information, it is indispensable that the coding of the data not be tied to a sole provider. The use of standard and open formats guarantees this free access, making possible the creation of compatible software.
To guarantee the perenniality of public data, it is indispensable that the use and maintenance of software does not depend on the good will of the providers, nor of monopolistic conditions, imposed by them. Systems can be guaranteed by the availability of the source code.
To guarantee national security it is vital to have systems that are devoid of elements that allow remote control or the transmission of non-desired information to third-parties. Therefore, it is required to have systems whose source code is freely accesable to the public, so that its inspection be allowed by the State, the citizens and a great number of freelance experts in the world.
Thus the first two are talking about open data formats, although the second states that having the source code is enough to guarantee preservation of data (which is what the translator undoubtedly meant when using the phrase "perenniality of data"). I would much rather have technical documentation of the data format than source code.
The third argument, about not having spyware, is a bit of a scare tactic, and I think the free software movement overstates the likelihood that experts will actually examine all source code. The bill expands on this theme later on, without pointing out that individual citizens will not be able to go in and modify the actual code that is running on government computers. They will have to take it on faith that the source code they are looking at is actually what is running on the computer. And of course the government could spy on its citizen's data no matter what kind of software it is running. This just removes the chance that the company that wrote the software is also going to spy on the data, which seems a very minor concern.
The bill goes on to point out the licensing benefits of free software, a paragraph that involves a bit of hand-waving:
The project clearly states that any given software in order to be acceptable for the State must not only be technically adequate to carry out a given task, but must also fulfill some requirements in license matters without which the State could not guarantee the citizen the adequate processing of data, the monitoring of its integrity, and the confidentiality and its permanent accessibility, all of which are critical elements for fulfillment of the project.
The bill then mentions that using free software could create jobs for local programmers, and also eliminate the issue of government liability if it is caught using unlicensed proprietary software (!). To its credit, while it does mention cost, it downplays it, and does admit there would be migration costs.
So what to do if no free software is available? The bill offers several choices, in descending order of desirability:
Microsoft response (original letter, translation, another translation)
On March 21, 2002, Juan Alberto Gonzalez, the General Manager of Microsoft Peru, sent a letter to Edgar Villanueva Nunez, the congressman who introduced Bill 1609.
The letter gives a very weak argument against Bill 1609, which reflects Microsoft position vis-a-vis open source back then--that open source would go away if Microsoft just put its weight behind a few token arguments. The letter claims Bill 1609 would be discriminatory. and does not consider "the dangers that this can bring from the point of view of security, guarantee, and possible violation of the intellectual property rights of third parties." It also claims the issue of cost is not important, and includes the following doozy: "One of the arguments behind the bill is the supposed freedom from costs of open-source software, compared with the costs of commercial software, without taking into account the fact that there exist types of volume licensing which can be highly advantageous for the State, as has happened in other countries." So Microsoft is basically saying sure we charge for software, but we may be able to charge you a bit less! How thoughtful.
Response from Edgar Villanueva Nunez (translation, another translation)
The letter from Microsoft was an easy target, and on April 8, 2002, Villanueva responded, in a long letter that shreds Microsoft's points in turn. He points out that the cost issue is a minor one (and in retrospect, he may wish he had left it out, since it just gives Microsoft ammunition). He also explains that the bill is talking about "free software", not "open source" (free software has to really be free, beyond just having the source code available -- the Free Software Foundation's website has a page that attempts to explain the difference).
Villanueva says the law would not be discriminatory (not being an expert on Peru's laws, I have no idea if this is true). He then points out a couple of issues with proprietary software that were not included in the preamble to the original bill: forced upgrades and cessation of tech support. "And as the whole infrastructure is based on proprietary data formats, the user stays 'trapped' in the need to continue using products from the same supplier, or to make the huge effort to change to another environment (probably also proprietary)."
To Microsoft's concern about "security, guarantee, and possible violation of the intellectual property rights of third parties", he responds:
I'm not sure about free software having fewer bugs, but he does a good job of reducing Microsoft's argument to ashes. These aren't arguments for using open data formats, but he is only bringing them up because Microsoft did.
He goes on to talk about support costs, since Microsoft brought that up (saying the cost of software is only 8% of the total cost of using it):
Now the use of free software contributes significantly to reduce the remaining life-cycle costs...This reduction in the costs of installation, support etc. can be noted in several areas: in the first place, the competitive service model of free software, support and maintenance for which can be freely contracted out to a range of suppliers competing on the grounds of quality and low cost. "
In the second place, due to the reproductive characteristics of the model, maintenance carried out for an application is easily replicable, without incurring large costs (that is, without paying more than once for the same thing) since modifications, if one wishes, can be incorporated in the common fund of knowledge.
Thirdly, the huge costs caused by non-functioning software ("blue screens of death", malicious code such as virus, worms, and trojans, exceptions, general protection faults and other well-known problems) are reduced considerably by using more stable software; and it is well known that one of the most notable virtues of free software is its stability.
This is actually a fairly weak part of his letter, but again he is only addressing it because Microsoft brought up the cost issue. I doubt the market for free software support is more competitive than that for Windows support, since the Windows market is bigger. The argument about bug fixes being shared may be true, but it also may be harder to not hear about fixes (unlike with proprietary software, where the fixes all come from one manufacturer who is aware who is running its software). And I think people should be wary of saying "open source software is better/faster/etc" since this is the kind of battle Microsoft can fight with its current software, and may just turn into a war between different research reports. As opposed to, say, open data formats, which Microsoft would have to change its policies to support and would have a much harder time arguing against.
Finally, Villanueva states:
We agree that in the private sector of the economy, it must be the market that decides which products to use, and no state interference is permissible there. However, in the case of the public sector, the reasoning is not the same: as we have already established, the state archives, handles, and transmits information which does not belong to it, but which is entrusted to it by citizens, who have no alternative under the rule of law. As a counterpart to this legal requirement, the State must take extreme measures to safeguard the integrity, confidentiality, and accessibility of this information.
Exactly! A great argument for open data formats.
One aspect of this lobbying was two letters sent from the American embassy [6/17/02 & 6/19/02]. One letter was from John R. Hamilton, the America ambassador to Peru, to Carlos E. Ferrero Costa, the President of the Congress, and cc'ed to Gloria G. Helfer Palacios, President of the Education and Technology Commission, and to Pedro A. Morales Mansilla, president of the Consumer Safety Commission. This letter claimed that the law could have a serious effect on the Peruvian software industry, which had the potential to create 15,000 jobs, and would also send a confusing message to foreign companies looking to invest in Peru. Enclosed, according to the letter, was a factsheet from Microsoft discussing its concerns over open source software. The second letter was from Stephen M. Liston, an economic advisor at the American embassy, to Pedro Morales, which was just a cover letter for another copy of the factsheet.
It was not officially proven that Microsoft was behind the letters, but it certainly looked suspicious. Around the same time, Microsoft also donated $550,000 in software and services [7/15/02] to the government of Peru. I exchanged some email with Jesus Marquina-Ulloa, webmaster of the GNU Peru site, who said that the lobbying succeeded: the bill was discussed in committee but never came to a vote, and is currently in a dormant state, although it may be revived in the future.
There was also an exchange of letters sent from the General Manager of the American Chamber of Commerce of Peru to the President of the Congress; the letter from AmCham Peru [5/27/02] and Villanueva's response [6/7/02] are online, but no translations are available.
A Babelfish translation of the letter gives the general gist, which is (i) to warn about migration costs, (ii) point out that software costs are a small part of the total cost, (iii) mention that government gets good deals on commercial software, and (iv) warn that free software does not have enough of a service industry to support the government . Plus the arguments about the law being discriminatory, hurting local companies and discouraging foreign investment. It also includes a somewhat dismissive response to the advantages that open data formats give to ensuring the preservation of data: "evidence does not exist that free software is superior to commercial software. On the contrary, it is possible to argue against free software on such criteria." Of course, he does not attempt to actually make such an argument.
A Babelfish translation of Villanueva's response shows that Villanueva merely pointed out that the letter had essentially the same points as the one sent by the General Manager of Microsoft Peru, and his response is to enclose a copy of that letter.
In January 2002, Richard Stallman wrote an editorial for LinuxToday about refusing Microsoft Word attachments [1/11/02]. His goal is to stop people from using proprietary software (Word), but his arguments about why closed data formats are bad are right on the money.
As it happened, right after I wrote the initial manifesto, I went down to Olympia, the capital of Washington, on Wednesday, January 30, 2002, to spend a day with the state PTSA lobbyist. I broached the idea of open data format legislation to my (at the time) two state representatives and one state senator. Note that all three are current or former Microsoft employees and the district they represent borders the one Microsoft's main campus is in and includes many Microsoft employees (in fact, shortly after that, I was redistricted out of their district, into the one that includes Microsoft's main campus). The responses were: From the Democratic representative: "interesting idea." From the Republican-but-secretly-Libertarian representative: "I don't think it should be a law, but I would need to think about it." From the Republican senator: "I would have a hard time voting for something like that."
The article discussed a three-part plan:
At the time, I felt that the laws should be the third step and that "ODFI compliance" would involve using the description and tools from the second step. My first discussions with politicians made it seem that I would need some technical heft behind the idea before it would ever be considered as a legal requirement.
However, in the time since I wrote the article several states and countries have had laws proposed to force governments to use only open source software. I would much prefer that they pass open data format laws instead. As a result, I am starting this site to start discussion about open data format laws (and open source laws), and hope to have an open data format law ready to be submitted to a state legislature in 2004.
About myself: my name is Adam Barr, and I worked as a developer at Microsoft for ten years, mostly on the NT kernel, before leaving in April 2000. I wrote a book about my experiences there titled Proudly Serving My Corporate Masters. Although I worked there and often defend Microsoft in discussions, I do not consider myself to be on Microsoft's "side". Opening up Microsoft's data formats is one of the goals of ODFI, and given Microsoft's response to open source laws, I suspect the company will also oppose open data format laws.